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CHAPTER  I 


INTRODUCTION 


Background 

The  goal  of  the  Air  Force  Institute  of  Technology 
(AFIT)  is  to  provide  the  education  necessary  to  meet  the 
technological  and  managerial  requirements  of  the  Air  Force 
(1:1).  The  Institute  accomplishes  its  goal  through  formal 
graduate  schools  and  Professional  Continuing  Education  (PCE) 
Programs.  AFIT  may  be  better  known  for  its  degree-granting 
Schools  of  Engineering  and  School  of  Systems  and  Logistics, 
but  by  far  the  greatest  number  of  students  are  involved  in 
professional  continuing  education. 

Within  the  School  of  Systems  and  Logistics,  the  PCE 
Program  seeks  to  provide  the  highest  quality,  most  up-to-date 
education  for  Air  Force  and  selected  Department  of  Defense 
(DoD)  logisticians.  The  Professional  Continuing  Education 
Program  consists  of  44  resident  and  many  nonresident  courses 
that  last  from  one  to  six  weeks  (1:126).  The  resident  pro¬ 
grams  attempt  to  maximize  the  learning  of  the  students  while 
they  are  at  Wright-Patterson  AFB  (where  the  school  is  located) 
in  order  to  minimize  the  time  students  are  away  from  their 
jobs.  Class  times  range  from  six  to  seven  hours  per  day,  five 
days  a  week.  For  these  courses,  AFIT  uses  guest  lecturers  who 
are  experienced  experts  working  throughout  the  base  complex  (2 


On-site,  seminars,  workshops,  and  correspondence 
courses  provide  nonresident  PCE .  For  example,  when  there  is 
an  adequate  group  of  people  at  one  location,  AFIT  instructors 
travel  to  these  students  in  order  to  hold  a  class  "on-site" 
(2:1).  This  has  the  added  advantage  of  keeping  the  instruc¬ 
tors  aware  of  current  problems  that  field  units  are  experien¬ 
cing. 

Because  of  the  rapid  growth  of  knowledge  and  constantly 
changing  technology,  neither  the  AFIT  faculty  or  facilities 
have  been  able  to  keep  pace  with  the  expanding  Air  Force  need 
for  Professional  Continuing  Education  (PCE)  (1:1).  For 
example,  the  School  of  Systems  and  Logistics  has  numerous  PCE 
courses  with  two-  or  three-year  backlogs.  A  need  also  exists 
for  new  courses  (2:1).  Facilities  and  monetary  restraints 
limit  the  growth  of  current  resident  and  nonresident  programs. 
This  growth  is  also  constrained  by  the  ceilings  on  the  hiring 
of  additional  qualified  instructors. 

Of  the  15,000  people  who  needed  resident  PCE  train¬ 
ing  in  the  1978-79  academic  year,  only  7,000  students  were 
able  to  receive  instruction  at  AFIT  (2:1).  During  this  same 
year,  nonresident  programs  reached  approximately  10,000  stu¬ 
dents.  Still  many  more  requests  exist  for  nonresident  educa¬ 
tion  than  AFIT  can  accommodate.  One  reason  for  this  increased 
need  is  the  decreasing  experience  level  in  the  military  and 
civil  service.  By  the  end  of  the  year,  the  percentage  of  Air 
Force  line  officers  who  will  be  newly  commissioned  lieutenants 
will  be  forty  percent  (4:2). 


For  these  reasons,  AFIT  needed  an  educational  deli¬ 
very  method  that  could  reach  more  students  with  existing 
courses  and  could  expand  to  cover  new  courses  without  signifi¬ 
cantly  increasing  either  the  number  of  faculty,  TDY  costs,  or 
resident  facilities.  In  1979,  Colonel  Lewis  Israelitt,  Dean 
of  the  School  of  Systems  and  Logistics,  devised  a  solution  to 
the  problem.  The  solution  centered  on  an  expansion  of  the 
telephone  educational  delivery  medium  already  being  used  by 
AFIT's  School  of  Systems  and  Logistics  and  the  School  of  Civil 
Engineering.  Both  schools  use  the  telephone  to  provide  short 
blocks  of  instruction  (one  or  two  hours  long)  to  remote  loca¬ 
tions.  The  terms  "tele-teach"  and  "tele- lecture"  were  selected 
to  refer  to  educational  delivery  modes  that  use  telephone  lines 
to  transmit  verbal  communication.  Using  this  medium,  AFIT 
instructors,  without  leaving  the  school,  routinely  lecture 
students  at  other  bases.  Commercial  dial-up  telephone  ser¬ 
vices  are  used  in  this  program  (2:2). 

The  use  of  tele -communications  is  not  a  new  concept. 
Since  the  early  1950s,  numerous  civilian  educational  institu¬ 
tions  have  operated  some  form  of  telephonic  instructional 
networks.  Today,  there  are  over  37  telephone  networks  that 
convey  instruction  to  college  students  attending  classes  away 
from  the  "home"  campus.  The  University  of  Wisconsin,  probably 
the  most  experienced  user  of  telephonic  networks,  serves  over 
35, COO  students  annually  via  its  educational  telephone  net¬ 
work.  Furthermore,  the  results  of  many  studies  indicate  no 
loss  of  learning  when  telephonic  instruction  is  compared  with 


traditional  resident  classroom  instruction  (2:2). 

Using  the  telephone  instructional  system  as  a  nucleus, 
AFIT  developed  a  system  named  the  Teleteach  Expanded  Delivery 
System  (TEDS).  The  instructors  felt,  because  of  the  technical 
nature  of  the  courses,  some  form  of  visual  delivery  mode  would 
be  essential  if  entire  courses  were  to  use  a  telephonic  system. 
Several  universities  and  colleges  were  contacted  and  published 
articles  were  reviewed  to  discover  the  most  successful  tech¬ 
niques  in  the  field  of  telephonic  delivery  systems.  The 
latest  electronic  instructional  aids  marketed  by  commercial 
companies  were  also  investigated.  Balancing  cost  and  effec¬ 
tive  learning  factors,  Colonel  Israelitt  selected  an  electronic 
blackboard  to  supplement  the  audio  system  component. 

The  Teleteach  Expanded  Delivery  Systems  gained  appro¬ 
val,  and  implementation  of  a  test  program  began  in  October 
1979.  Two  nationwide  networks  were  installed  to  connect 
remote  classrooms  with  a  resident  class  at  AFIT.  Each  network 
has  two  pairs  of  dedicated  telephone  lines.  One  line  pair 
transmits  two-way  audio  communication;  the  second  pair  trans¬ 
mits  writing  on  the  electronic  blackboard.  While  no  visual 
contact  between  the  remote  class  and  the  instructor  is  pro¬ 
vided,  the  electronic  blackboard  offers  a  significant  capa¬ 
bility  for  written  communication.  Words  or  equations  placed 
on  the  special  blackboards  are  digitized,  transmitted  over  the 
telephone  lines,  and  reproduced  on  television  monitors  located 
in  each  classroom.  Moreover,  microphones  are  provided  for 
students,  instructors,  and  monitors  at  each  location. 
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Therefore,  oral  and  blackboard  interaction  of  all  participants 
is  possible  (5:5-9) . 

The  primary  users  of  AFIT's  PCE  courses  are  the  Air 
Force  Systems  Command  (AFSC)  and  the  Air  Force  Logistics  Com¬ 
mand  (AFLC) .  Both  commands  have  major  centers  geographically 
located  throughout  the  continental  United  States  including 
Wright-Patterson  AFB.  The  telephonic  networks  connect  selected 
bases  of  each  command  with  AFIT.  One  network  consists  of  five 
Air  Logistics  Centers  of  AFLC.  The  other  links  four  AFSC 
locations.  Since  these  are  separate  networks,  AFIT  may  offer 
courses  tailored  for  each  command  simultaneously  (6:6).  In 
this  manner,  one  instructor  using  TEDS  may  conduct  a  course 
in  an  AFIT  (WPAFB)  classroom  while  simultaneously  having  addi¬ 
tional  students  at  different  remote  sites.  AFSC  and  AFLC 
select  students  for  the  local  class  from  Wright-Patterson 
personnel. 

Several  other  features  are  incorporated  into  the 
system.  Both  audio  and  graphics  are  recorded  on  a  stereo- tape 
recorder.  The  tapes  can  be  replayed  both  at  Wright-Patterson 
AFB  (WPAFB)  or  any  remote  site.  Video  tape,  35  mm  slides, 
vugraphs,  and  written  materials,  when  necessary,  are  distri¬ 
buted  to  each  site  (4:6). 

While  telephonic  instruction  is  a  proven  effective 
method  of  learning,  several  distinct  features  in  TEDS  preclude 
a  direct  correlation  with  other  systems.  First,  most  re¬ 
search  of  telephonic  delivery  systems  has  been  based  on  one- 
or  two-hour  classes  meeting  one  or  twice  a  week.  The 


intensified  AFIT  program  operates  a  four-hour  daily  schedule 
with  ten-minute  breaks  between  classes.  No  more  than  four 
hours  were  practical  because  time  zone  differences  demanded 
the  resident  TEDS  class  at  Wright-Patterson  AFB  be  conducted 
from  twelve  o'clock  to  four  o'clock  in  order  that  West  Coast 
classes  could  be  conducted  during  normal  duty  hours.  Second, 
unlike  most  reported  research,  AFIT  often  uses  guest  lec¬ 
turers  who,  though  experts  in  their  specialty,  are  not  pro¬ 
fessional  educators.  Third,  TEDS  incorporates  the  electronic 
blackboard,  which  was  not  a  part  of  other  systems  researched. 
Other  factors  such  as  mandatory  attendance,  different  presen¬ 
tation  formats,  and  inexperience  with  the  medium  distinguish 
TEDS  from  systems  previously  evaluated  (2:2-3). 

Justification  of  Research 

Any  innovative  project  of  this  size,  cost,  and  student 
impact  must  be  rigorously  evaluated  by  independent  researchers 
before  being  fully  implemented.  However,  the  already  sizeable 
requirements  for  Air  Force  and  DoD  Professional  Continuing 
Education  courses  dictated  that  AFIT  move  immediately  to  ex¬ 
pand  its  enrollments.  Hence,  TEDS  was  developed  and  integrated 
into  the  PCE  program  without  prior  evaluation  in  order  to  sat¬ 
isfy  increasing  requirements  while  meeting  budgetary  and  man¬ 
power  restraints.  Considerable  research  is  now  underway  to 
evaluate  the  effectiveness  of  TEDS. 

Summary  of  Current  Research 


Dr.  G.  Ronald  Christopher,  Chief  of  the  Plans  and 


Evaluation  Division  at  AFIT,  is  conducting  the  evaluation  of 
the  Teleteach  Expanded  Delivery  System.  It  is  important  that 
the  strengths  and  weaknesses  of  this  delivery  mode  be  revealed 
through  controlled  research  (3). 

Three  studies  have  already  been  completed  which  eval¬ 
uated  some  of  the  courses  presented  during  the  TEDS  pilot 
program.  Lieutenant  Colonel  Charles  W.  McNichols  analyzed 
SYS  123  (Fundamentals  of  Acquisition  Management);  Major  Alvin 
L.  Milam  and  G.  Ronald  Christopher  analyzed  a  second  offering 
of  SYS  123;  and  Captains  David  E.  Fortna  and  Ronne  G.  Mercer 
recently  completed  their  formal  thesis  evaluating  LOG  220 
(AFLC  Materiel  Management  classes  79E  and  80A) . 

The  research  findings  from  each  separate  study  will 
provide  the  comprehensive  data  necessary  for  a  complete  evalu¬ 
ation  of  TEDS.  Conclusions  and  recommendations  arising  from 
these  independent  studies  will  provide  the  guidance  necessary 
to  determine  the  future  of  TEDS.  A  summary  of  the  results  of 
the  three  studies  will  now  be  presented. 

Lieutenant  Colonel  McNichols  evaluated  the  first 
offering  of  SYS  123  (Fundamentals  of  Acquisition  Management) . 
This  course  was  originally  named  SYS  326.  The  purpose  of  this 
AFIT  technical  report  was  to  answer  three  of  the  original 
research  questions  of  Dr.  Christopher's  evaluation  of  TEDS. 
These  questions  were: 

1.  Are  student  groups  (control/experiment)  comparable 
in  terms  of  education  level,  grade/rank,  age,  and  entry 
level  knowledge? 

2.  What  differences  in  academic  achievement  occurred 
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between  resident  student  groups  receiving  instruction 
face-to-face  with  the  presenter  and  student  groups  re¬ 
ceiving  instruction  without  face-to-face  presentations 
when  both  groups  used  the  TEDS? 

3.  To  what  extent  was  the  TEDS  acceptable  to  students, 
their  supervisors,  presenters,  visitors,  and  site  moni¬ 
tors?  (Only  student  acceptability  is  addressed  in  this 
report . )  [6:3-4]. 

The  author  concluded  that: 

1.  The  four  student  groups  (WPAFB ,  S.4MS0 ,  Eglin,  and 
Hanscom)  attending  the  October  1979  course  offering  of 
SYS  123  were  statistically  equivalent  in  terms  of  demo¬ 
graphic  variables.  (A  later  review  revealed  a  signi¬ 
ficant  difference  with  respect  to  the  demographic 
variable  rank/grade.  Also  the  groups  were  found  to 
differ  in  terms  of  entry-level  knowledge  with  the  re¬ 
mote  group  scoring  higher.) 

2.  Statistically  significant  differences  in  mean  post¬ 
test  grades  were  found  between  resident  and  remote  site 
students.  The  mean  post-test  grade  was  63. 2%  at  the 
resident  site  and  57.51  at  the  remote  sites.  The  prac¬ 
tical  significance  of  a  roughly  6?  difference  in  mean 
test  grade  is,  however,  hard  to  judge.  In  comparing 
achievement  as  measured  by  the  difference  between  pre¬ 
test  and  post-test  grades  for  each  student,  a  larger 
(and  statistically  more  significant)  difference  was 
found  between  the  resident  and  remote  sites  with  a  mean 
difference  of  42.8%  at  the  resident  site  and  28.5%  at 
the  remote  sites.  This  result  suggests  that  students 
with  a  low  starting  level  of  knowledge  may  increase 
their  knowledge  level  by  a  greater  amount  in  the  face- 
to-face  situation  than  they  do  in  the  teleteach  remote 
situation.  The  impact  of  demographic  differences  on 
student  learning  as  measured  by  either  post-test  grade 
or  post- test/pretest  grade  difference  was  overshadowed 
by  the  impact  attributable  to  location  at  which  the 
course  was  taken. 

3.  Teleteach  acceptability  varied  widely  among  students 
at  various  remote  sites,  ranging  from  quite  positive  at 
SAMSO  to  quite  negative  at  Eglin.  Significantly  more 
positive  opinions  of  course  value,  structure,  and  con¬ 
tent  were  reported  by  students  receiving  the  instruction 
in  the  face-to-face  mode.  The  data  suggest  that  class¬ 
room  facilities,  which  apparently  differed  in  quality 
among  the  remote  sites,  may  have  an  important  impact 

on  student  attitudes  about  the  course  and  the  accepta¬ 
bility  of  the  teleteach  delivery  system  [6:20-21]. 
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Drs.  Milam  and  Christopher  prepared  the  second  AFIT 
technical  report  which  evaluated  the  second  TEDS  offering  of 
SYS  123.  The  research  objectives  were  the  same  as  for  the 
first  report.  Based  on  the  January  1980  TEDS  offering  of 
SYS  123,  the  authors  found: 

1.  The  resident  and  remote  site  students  were  statis¬ 
tically  different  in  terms  of  rank/grade  (p  =  .0068) 
and  age  (p  =  .0177)  with  the  remote  site  students  being 
of  higher  rank/grade  and  older.  The  remote  site  stu¬ 
dents  were  slightly  more  educated  and  somewhat  more 
experienced.  In  terms  of  entry  level  knowledge,  the 
five  sites  were  statistically  equivalent. 

2.  All  five  sites  were  statistically  equivalent  in 
terms  of  achievement.  In  looking  at  the  post-test 
grade  for  all  five  sites,  the  only  predictor  is  the 
educational  level  of  the  student.  In  terms  of  achieve¬ 
ment  for  all  five  sites,  the  only  predictor  is  years 

of  experience  and  this  appears  to  have  a  negative  in¬ 
fluence,  i.e.,  the  more  experience,  the  less  achieve¬ 
ment  . 

3.  Teleteach  acceptability  varied  significantly  among 
the  five  sites  with  HQ  (Andrews  AFB)  being  the  most 
positive  and  AD  (Eglin  AFB)  being  the  most  negative. 
Significant  statistical  differences  were  found  among 
all  sites.  The  most  significant  predictor  of  Teleteach 
acceptability  was  site  location.  As  indicated  earlier, 
this  could  possibly  be  attributed  to  a  variety  of 
feelings  about  the  course,  classroom,  and  method  of 
delivery  [7:15]. 

A  third  study  evaluating  TEDS  was  a  formal  thesis  by 
Captains  Fortna  and  Mercer.  Their  purposes  were  to  determine 
whether  TEDS  was  as  effective  educationally  as  resident  in¬ 
struction  of  the  same  course,  whether  TEDS  was  an  acceptable 
mode  of  learning  for  both  students  and  instructors,  and 
whether  TEDS  was  less  costly  on  a  per  student  basis  than  the 
previous  delivery  system.  In  their  research,  the  authors 
evaluated  the  July  1979  nonTEDS  and  the  October  1979  TEDS 
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offerings  of  LOG  220  (Materiel  Management) . 

The  authors  concluded  that  for  the  LOG  220  course: 

1.  TEDS  was  as  effective  educationally  as  resident 
instruction . 

2.  TEDS  was  acceptable  to  participating  students 
and  instructors. 

3.  TEDS  was  more  economical  on  a  per  student  basis 
than  resident  instruction  (4:93-100). 

The  thesis  also  contained  a  review  of  selected  studies 
by  professional  educators  who  have  examined  various  telephonic 
methods  of  instructional  delivery  in  terms  of  their  relative 
success.  In  general,  these  studies  supported  the  contention 
that  telephonic  instruction  is  a  viable  means  of  communicating 
knowledge  to  a  large  body  of  people. 

The  main  points  summarized  in  the  thesis  were  as 

follows : 

First,  as  expressed  by  Edelman,  the  telephonic  delivery 
system  appears  to  be  a  viable  solution  to  the  conflict 
between  high  demand  for  education  and  low  resource 
availability.  Second,  Edelman,  Dotterweich,  Arndt  and 
Weinswig,  Weinstock,  and  Reid  all  contend  that  attitude 
--both  student  and  instructor  attitude- -will  determine 
how  effective  this  type  of  system  is.  Third,  many 
evaluators,  including  Edelman  and  others  cited  through¬ 
out  this  study,  have  tested  the  learning  effectiveness 
of  this  delivery  system  as  a  basis  for  determining 
overall  effectiveness.  Fourth,  the  other  major  measure 
of  effectiveness  used  by  authors  such  as  Rao  and  Hicks 
focuses  on  the  cost  of  this  type  of  system  as  compared 
to  the  in-residence  methodology.  This  comparison  is 
exceptionally  relevant  to  AFIT’s  TEDS  in  light  of  the 
Air  Force's  limited  resources  and  its  desire  to  realize 
maximum  benefit  from  each  dollar  spent  for  education 
[4:22]  . 


In  addition  to  the  research  already  published,  two 
other  technical  reports  on  courses  using  TEDS  were  near  com¬ 
pletion  at  the  time  this  thesis  was  written.  Mr.  Jeffrey  C. 
Daneman  was  analyzing  QMT  170  (Principles  of  Contract  Pricing) 
and  Major  Alvin  L.  Milam  and  Dr.  Christopher  were  analyzing 
a  second  TEDS  offering  of  LOG  220  (AFLC  Materiel  Management) . 

Statement  of  the  Problem 

A  requirement  exists  for  an  independent  research 
effort  to  evaluate  the  TEDS  method  of  delivery  compared  to 
the  traditional  one  instructor-one  class  nonTEDS  method  of 
instruction  for  each  course  in  the  experiment. 

Specifically,  this  thesis  evaluates  the  use  of  TEDS 
in  the  1980  offerings  of  the  System  Program  Management  course 
(SYS  223).  The  general  areas  addressed  are:  1)  the  learning 
effectiveness  of  TEDS  versus  nonTEDS  instruction  and  2)  the 
degree  of  system  acceptance  by  students  and  instructors  of 
this  method  of  instruction. 

Research  Objectives 

To  make  this  analysis,  the  following  research  ques¬ 
tions  from  the  original  TEDS  experiment  were  addressed  (2:9): 

1.  Are  student  groups  (classes)  comparable  in  terms 
of  the  demographic  variables  of  education  level,  grade/rank, 
age,  and  entrv  level  knowledge?  (Although  not  mentioned  in 
the  original  objective,  years  of  experience  in  a  job  related 
to  the  course  is  also  analyzed.) 

2.  What  effect  upon  academic  achievement  did  the 
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TEDS  course  offerings  have  compared  to  the  nonTEDS  offering 
of  the  same  course? 

3.  What  difference  in  academic  achievement  were 
there  between  the  two  TEDS  offerings  of  the  same  course? 

4.  To  what  extent  was  the  TEDS  acceptable  to  stu¬ 
dents  and  instructors? 

5.  To  what  extent  did  students  consider  the  TEDS 
schedule  acceptable? 

Scope 

This  evaluation  includes  three  offerings  of  SYS  223. 
Two  TEDS  offerings  were  presented  through  the  Air  Force 
Systems  Command  (AFSC)  network.  A  resident  nonTEDS  offering 
( 80BR)  constituted  the  nonTEDS  group  against  which  the  two 
TEDS  offerings  (80CT  and  80DT)  were  compared.  The  nonTEDS 
offering  was  conducted  from  17  March  through  18  April  1980. 
The  80CT  TEDS  offering  of  SYS  223  from  21  April  through  6 
June  1980  was  the  first  TEDS  course.  The  80DT  offering  of 
SYS  223  from  23  June  through  8  August  1980  was  the  second 
offering  using  the  TEDS  mode  of  delivery.  Each  TEDS  class 
was  further  divided  into  the  resident  TEDS  class  and  remote 
TEDS  classes.  In  this  thesis,  "Resident"  will  refer  to  TEDS 
classes  taught  at  WPAFB .  Although  accurate,  it  will  not  be 
used  to  refer  to  the  nonTEDS  class  taught  at  WPAFB. 

Since  this  analysis  contrasts  data  from  only  three 
offerings  of  SYS  223,  generalizations  from  these  findings 
apply  only  to  these  three  offerings  and  do  not  necessarily 
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apply  to  SYS  223  overall  or  the  TEDS  instructional  mode  in  toto. 

The  methodology  (to  be  discussed  in  Chapter  II)  is 
very  similar  to  that  used  in  previous  studies  of  other  courses 
so  that  further  aggregate  analysis  could  be  possible  in  the 
overall  evaluation.  The  methodology  was  derived  directly 
from  the  research  efforts  listed  earlier. 

Hypotheses 

In  order  to  test  learning  effectiveness  and  TEDS 
acceptability,  the  following  hypotheses  were  constructed: 

1.  There  is  no  significant  difference  in  the  age 
category  among  the  student  comparison  groups. 

2.  There  is  no  significant  difference  in  the  grade/ 
rank  category  among  the  student  comparison  groups. 

3.  There  is  no  significant  difference  in  the  educa¬ 
tion  level  category  among  the  student  comparison  groups. 

4.  There  is  no  significant  difference  in  the 
experience  category  among  the  student  comparison  groups. 

5.  There  is  no  significant  difference  between  the 
mean  acceptance  scores  of  the  various  combinations  of  student 
groups . 

6.  There  is  no  significant  difference  between  stu¬ 
dent  comparison  group  means  on  either  post-test  scores  or 
achievement . 

7.  Instructors  for  SYS  223  consider  TEDS  an  accept¬ 


able  instructional  delivery  system. 


CHAPTER  II 


METHODOLOGY 

The  methodology  used  in  evaluating  the  hypotheses 
constructed  in  Chapter  I  is  developed  in  this  chapter.  It 
begins  with  a  discussion  of  the  sampling  plan  and  student 
and  faculty  populations.  This  is  followed  by  an  explanation 
of  the  experimental  design  and  statistical  procedures.  Then 
decision  criteria  were  established  to  determine  whether  the 
results  of  the  data  analysis  supported  the  hypotheses.  The 
set  of  assumptions  necessary  to  permit  use  of  the  statistical 
procedures  conclude  the  chapter. 

Sampling  Plan 

The  sampling  plan  consisted  of  a  comparison  among 
three  separate  offerings  of  SYS  223  (System  Program  Manage¬ 
ment)  .  As  described  in  Chapter  I  (under  Scope)  ,  the  TEDS 
80CT  and  80DT  offerings  were  compared  to  the  nonTEDS  80BR 
offering  of  SYS  223. 

Each  TEDS  offering  consisted  of  a  resident  class  at 
WPAFB  and  interaction  with  four  remote  classes.  These  four 
remote  sites  were  located  at: 

1.  Electronic  Systems  Division  (ESD)  ,  Hanscom  AFB, 
Massachusetts 

2.  Armament  Division  (AD),  Eglin  AFB,  Florida 
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3.  Space  Division  (SD) ,  Los  Angeles,  California 

4.  AFSC  Headquarters  (HQ),  Andrews  AFB,  Maryland 


Student  and  Faculty  Populations 

The  sample  data  were  drawn  from  student  and  instructor 
populations.  The  universe  from  which  the  student  sample  was 
taken  consisted  of  all  military  and  civilian  personnel  employed 
by  AFSC  who  were  also  eligible  to  participate  in  PCE  courses. 
The  student  population  of  interest  included  AFSC  personnel  who 
were  eligible  for  enrollment  in  the  1980  offerings  of  SYS  223. 
Eligible  students  were  either  military  officers  between  first 
lieutenant  and  lieutenant  colonel,  military  enlisted  grades 
E-4  to  E-9,  and  civilians  of  grade  09  and  above.  Students 
were  required  to  have  program  office  experience  and  current 
or  pending  assignment  to  a  program  office  or  staff  position 
involved  in  system  program  (acquisition)  management. 

The  universe  from  which  the  instructor/lecturer  sample 
was  taken  consisted  of  civilian  and  military  AFIT  faculty  and 
guest  lecturers  who  were  qualified  to  present  lessons  in  the 
SYS  223  courses.  The  faculty  population  of  interest  were 
those  AFIT  instructors  and  guest  lecturers  who  were  selected 
to  make  presentations  in  SYS  223. 

A  convenience  sample  of  students  was  selected  which 
included  students  enrolled  in  the  SYS  223,  Systems  Program 
Management  courses,  between  17  March  and  15  August  1980.  Data 
collected  from  three  course  offerings  during  that  time  frame 
were  used  to  compare  nonTEDS,  resident  TEDS,  and  remote  TEDS 
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classes.  The  group  of  48  students  taking  SYS  223  in  class 
80ER  in  residence  at  Wright  -  Patterson  AFB  constituted  the 
nonTEDS  class. 


The  80CT  and  the  80DT  offerings  of  SYS  223  consti¬ 
tuted  the  TEDS  classes  totaling  155  students.  Class  80CT  had 
24  students  in  the  resident  TEDS  Wright-Patterson  AFB  class, 
with  64  students  at  remote  AFSC  sites.  Class  80DT  had  22 
students  in  its  resident  portion  of  the  TEDS  class,  with  45 
students  at  the  same  remote  sites  as  80CT. 

There  were  no  known  biasing  variables  in  the  student 
selection  process.  The  student  selection  process  was  the 
responsibility  of  AFSC  and  was  assumed  to  have  been  random. 

Data  Collection 

Demographic  Data.  During  the  first  session  of  each 
class,  students  completed  a  demographic  data  collection 
instrument  which  recorded  students'  rank/grade,  educational 
achievement  level,  age,  and  years  of  experience  in  a  job- 
related  to  the  course.  These  data  were  used  to  compare  the 
homogeneity  of  the  separate  classes. 

The  demographic  data  were  grouped  into  categories, 
and  each  category  was  assigned  an  ordinal  ranking  for  purposes 
of  comparison. 

End-of-Course  and  Instructor  Data.  Students  com¬ 
pleted  end-of -course  questionnaires  during  the  final  class 
day.  The  questionnaires  acquired  student  opinions  concerning 
TEDS  acceptability,  course  value,  course  structure,  instructor 
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performance,  etc.  (Appendix  B) . 


Instructors'  attitudes  toward  acceptability  of  TEDS 
were  collected  via  an  AFIT-developed  instructor  critique 
questionnaire  (Appendix  C) .  Instructor  opinions  about  TEDS 
were  limited  to  those  instructors  and  guest  lecturers  actu¬ 
ally  involved  in  either  SYS  223  80CT  or  80DT  offering. 
Unfortunately,  the  researcher  had  no  control  over  the  gather¬ 
ing  of  this  data.  A  six-month  delay  was  incurred  before  some 
instructors  were  asked  to  fill  out  these  questionnaires. 
Therefore,  the  data  were  incomplete  and  the  results  were  less 
meaningful . 

Pre-test  Scores.  A  pre-test  developed  by  the  SYS  223 
course  director  was  administered  during  the  first  session  of 
each  class.  This  test  covered  a  wide  range  of  system  acqui¬ 
sition  and  program  management  topics.  This  test  served  to 
identify  student  entry-level  knowledge.  Students  were  re¬ 
quested  not  to  guess.  Pre-test  scores  were  used  as  a  factor 
for  determination  of  the  homogeneity  of  classes  and  as  a  base¬ 
line  data  for  comparison  of  student  learning. 

Post-test  Scores.  Content  tests  (or  quizzes)  were 
developed  by  the  course  director  and  administered  periodically 
throughout  the  course.  These  test  scores  were  combined  and 
averaged  for  each  student.  The  combined  average  was  then 
identified  as  the  post-test.  Class  80BR  had  two  tests,  while 
classes  80CT  and  80DT  each  had  three  tests  which  were  included 
in  this  comparison.  Class  80BR  had  only  two  tests  since  they 
met  for  six  hours  per  day  and  covered  the  material  at  a 
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faster  rate.  The  TEDS  classes  (80CT  and  80DT)  had  five  tests 
Only  the  first  three  were  in  this  analysis  since  they  cover 
the  same  material  as  was  tested  in  class  80BR. 

Post-test  Scores  Minus  Pre-test  Scores.  The  differ¬ 
ence  between  the  mean  post-test  scores  and  the  mean  pre-test 
scores  was  used  as  the  measure  of  achievement.  These  achieve 
ment  measures  (heretofore  referred  to  as  achievement  score) 
were  used  as  a  basis  for  comparison  among  the  nonTEDS ,  resi¬ 
dent  TEDS,  and  remote  TEDS  classes. 

Data  Base.  Student  responses  and  test  scores  were 
collected  using  student  completed  standard  computer  answer 
sheet  A,  Optical  Scanning  Form  D  51120-A.  These  scan  sheets 
were  processed  on  the  AFLC  CREATE  computer  system  using  AFIT' 
OPSCAN  equipment  located  in  the  School  of  Systems  and  Logis¬ 
tics.  A  numeric  student  identification  code  permitted  con¬ 
solidation  of  data  items  for  each  student. 

After  processing,  output  data  in  punch  card  form 
were  loaded  into  files  on  the  ASD  CYBER  computer  system  to 
prepare  a  consolidated  data  base.  A  program  was  run  which 
consolidated  test  scores,  demographic  data,  and  end-of -course 
critique  data  by  individual  student.  The  result  of  this  data 
consolidation  process  is  shown  in  Appendix  C.  The  format  of 
each  data  record  is  as  follows: 

Record  Columns  Contents 

1  Student  locations:  1-WPAFB;  7-SD;  8-AD; 

9-ESD,  0-HQ 

2-4  Student  number 
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Record  Columns 

Contents 

5-17 

Demographic  questions  1-13 

25-54 

End-of-course  critique  items  1-30 

56-58 

Pre-test  score  (percent) 
nearest  .1 

rounded 

to 

60-62 

Post-test  score  (percent) 
nearest  .1 

rounded 

to 

64-66 

Post-test  score  (percent) 
nearest  .1 

rounded 

to 

68-70 

Post-test  score  (percent)  rounded 
nearest  .1  (Class  80BT  had  only 
post-tests) 

to 

two 

In  the  data  base,  the  alpha  responses  from  the  demo¬ 
graphic  instrument  and  end-of-course  critique  forms  were 
transformed  to  a  numeric  representation.  For  example,  "A"  stu¬ 
dent  responses  were  assigned  a  value  of  0,  "B"  responses  a  1, 
etc.  A  blank  column  was  used  to  represen"  missing  responses. 
The  computer  list  containing  these  records  is  included  as 
Appendix  E. 

Experimental  Design 

The  experimental  design  consisted  of  a  comparison  of 
demographic  data,  end-of-course  critique  data,  and  test  score 
data  in  various  combinations.  Table  1  displays  the  combina¬ 
tions  examined.  Table  2  delineates  the  specific  areas  of 
evaluation  used  in  this  study. 

Statistical  Procedures 

Statistical  methodologies  selected  from  the  Statisti¬ 
cal  Package  for  the  Social  Sciences  (SPSS)  were  applied  to 
this  analysis.  These  procedures  included  crosstabulation  and 
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TABLE  1 


EXPERIMENTAL  DESIGN 


Combinat ions 
for 

Comparison 

Demographic  Data/ 
Test  Score  Data/ 
End-of-Course  Data 

Acceptance 

of 

TEDS 

1. 

nonTEDS  BR 

to 

TEDS  CT 

X 

2. 

nonTEDS  BR 

to 

TEDS  DT 

X 

3. 

nonTEDS  BR 
TEDS  CT 

to 

Resident 

X 

4. 

nonTEDS  BR 
TEDS  DT 

to 

Resident 

X 

5. 

Resident  TEDS  CT  to 
Resident  TEDS  DT 

X 

X 

6. 

TEDS  CT  to 

TEDS  DT 

X 

X 

7. 

Remotes  of 
Remotes 

TEDS  CT  to 
of  TEDS  DT 

X 

X 

Chi- 

square,  one- 

way 

analysis- 

of -variance 

(ANOVA)  ,  and  regres 

sion  analysis. 

Crosstabulation  and  Chi-square  (contingency  table) 
Analysis .  Crosstabulation,  a  common  descriptive  technique, 
was  used  to  discover  how  two  categorical  variables  were  asso¬ 
ciated.  With  this  approach,  items  were  displayed  in  a  two-way 
categorization  to  permit  comparison  of  the  percentage  distri¬ 
bution  of  responses  to  one  item  across  categories  defined  by 
a  second  item.  This  technique  was  particularly  useful  in 
comparing  demographic  and  end-of-course  items  among  the  various 
treatments  under  evaluation  (9:39). 

Each  of  the  frequency  distributions  irrayed  in  this 
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TABLE  2 


AREAS  OF  EVALUATION 


Demo-  Test  End-of -Course 

graphic  Score  Critique 

Data  Data  Data 


Rank  X 
Education  Level  X 
Age  X 
Experience  X 
Pre-test  Scores 


Post-test  Scores 
Academic  Achievement 
TEDS  Related 
Remote  TEDS  Peculiar 
TEDS  Acceptance 
Schedule  Acceptance 


X 

X 

X 


X 


X 

X 

X 


comparison  were  tested  by  use  of  the  Chi-square  statistic. 

This  statistic  was  calculated  from  the  data  array  and  was  con¬ 
cerned  with  the  question  of  statistical  independence  of  the 
two  modes  of  classification.  The  hypothesis  was  that  the  two 
modes  of  classification  (item  and  site  or  course  offering,  in 
this  application)  were  statistically  independent.  Rejection 
of  the  hypothesis  would  then  suggest  that  the  variables  were 
not  independent  (thus  dependent)  and  would  imply  statistically 
significant  differences  among  the  course  offerings.  This 
technique  was  used  to  compare  demographic  and  end-of -course 
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items  between  nonTEDS  classes  and  the  TEDS  classes.  To  faci¬ 
litate  comparison  of  end-of -course  items,  "strongly  agree" 
and  "agree"  responses  were  combined  to  form  a  single  "agree" 
variable.  Similarly,  the  "strongly  disagree"  and  "disagree" 
answers  were  combined  to  form  a  single  "disagree"  variable. 

For  the  remainder  of  this  thesis,  these  statements  will  be 
referred  to  as  "questions".  "0"  corresponded  to  "strongly 
agree"  and  "4"  corresponded  to  "strongly  disagree". 

One-Way  Analysis  of  Variance  (ANOVA).  The  one-way 
analysis  of  variance  (ANOVA)  procedure  was  used  to  examine 
the  question  of  whether  or  not  two  classes  (sites)  were  dif¬ 
ferent  with  respect  to  their  mean  value  (9:424).  The  fixed 
effects  model  was  used  because  inferences  were  made  only  from 
those  categories  included  and  because  the  research  considered 
the  given  groups  of  a  factor  to  be  fixed  over  repeated  experi¬ 
ments  (9 : 399)  . 

The  ANOVA  technique  presented  a  mean  score  or  single 
criterion  variable  for  each  group  of  respondents.  Then  a 
test  of  the  hypothesis  that  all  group  means  were  equal  was 
possible.  Rejection  of  this  hypothesis  implied  that  at  least 
two  of  the  groups  differed  significantly  in  criterion  mean 
score.  For  example,  this  method  was  used  to  examine  the 
following  variables. 

1.  Pre-test  score 

2.  Post-test  score 

3.  Achievement  (post-test  minus  pre-test  score) 

4.  Acceptance 


tk 


Analyses  were  made  between  the  WPAFB  nonTEDS  class  and: 


1. 

2. 

3. 

4. 

To  complete 
1. 


TEDS  SYS  223  80CT 
TEDS  SYS  223  80DT 

Resident  TEDS  SYS  223  CT  only,  and 
Resident  TEDS  SYS  223  DT  only 
the  analyses,  comparisons  were  made  between: 
Pooled  remote  classes  of  TEDS  SYS  223  CT  versus 


TEDS  SYS  223  DT 


2.  Resident  TEDS  SYS  223  CT  versus  resident  TEDS 
SYS  223  DT  only 

3.  Remotes  of  TEDS  SYS  223  CT  versus  remotes  of 
TEDS  SYS  223  DT 


Stepwise  Multiple  Regression  Analysis.  Stepwise  mul¬ 
tiple  regression  was  performed  in  an  attempt  to  explain  or 
identify  any  relationship  between  selected  variables.  This 
methodology  assumes  that  the  values  of  one  variable,  a  depend¬ 
ent  variable,  can  be  predicted  or  explained  by  the  values  of 
the  independent  variables  under  consideration.  The  method  is 
based  upon  the  statistical  technique  of  "ordinary  least 
squares"  (9:320-322). 

This  heuristic  regression  analysis  sequentially 
selected  specified  independent  variables  to  be  used  as  pre¬ 
dictors  of  the  dependent  variable.  The  sequence  corresponds 
to  the  prediction  ability  of  the  independent  variables. 
Appropriate  demographic  variables,  attitudinal  variables, 
test  scores,  locations,  and  mode  of  presentation  were  examined 
to  determine  any  possible  value  as  predictors  of  the  learning 
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factors  (post-test  scores  and  academic  achievement)  and 
acceptance  of  TEDS.  Only  those  predictor  variables  that  were 
statistically  significant  at  the  five  percent  level  were  con¬ 
sidered  part  of  the  linear  regression  equation. 

Likert  Scale.  A  Likert  scale  using  numerical  values 
ranging  from  zero  to  four  was  used  to  code  responses  of  selected 
statements  chosen  from  the  end-of-course  critiques.  Statistical 
tests  and  class  comparison  were  performed  on  the  resulting 
variables . 

Decision  Criteria 

In  addition  to  the  statistical  tests  applied  to  the 
research  data,  decision  rules  were  used  to  determine  whether 
the  results  of  the  data  analysis  satisfied  the  research  objec¬ 
tives  . 

To  determine  whether  the  student  groups  were  compar¬ 
able  in  terms  of  the  demographic  variables,  the  following 
decision  rule  was  established:  If  the  contingency  table 
analysis  yields  no  statistical  difference  (at  the  .05  level) 
between  the  comparison  groups,  accept  the  null  hypothesis 
that  there  is  no  relationship  between  the  groups. 

To  determine  whether  the  research  objective  of  learn¬ 
ing  effectiveness  was  met  with  the  implementation  of  TEDS  in 
the  SYS  223  course,  the  following  decision  rule  was  established: 
If  there  is  no  statistically  significant  difference  in  the  post¬ 
test  minus  pre-test  scores  (achievement  scores)  of  the  nonTEDS 
class  and  the  TEDS  class,  accept  the  null  hypothesis  that 


learning  by  the  TEDS  method  was  as  effective  as  learning  by 
the  nonTEDS  method  of  instruction. 

To  determine  whether  the  research  objective  of  stu¬ 
dent  acceptability  of  the  TEDS  method  was  met  in  the  SYS  223 
course,  this  decision  rule  was  set:  If  the  mean  values  of 
the  questions  selected  to  determine  acceptability  (questions 
23  and  25  from  the  students'  end-of-course  critique)  indicate 
that  a  majority  of  participants  found  TEDS  acceptable,  accept 
the  null  hypothesis  that  the  TEDS  approach  used  in  SYS  223 
was  acceptable  to  participating  students. 

To  determine  whether  the  research  objective  of  TEDS 
schedule  acceptability  was  met,  the  following  decision  rule 
was  set:  If  a  group  response  was  greater  than  50  percent  in 
agreement  (disagreement)  with  question  17,  conclude  that  the 
TEDS  schedule  was  acceptable  (unacceptable) .  Other  end-of- 
course  questions  were  evaluated  in  a  similar  manner. 

To  determine  whether  the  research  objective  of  in¬ 
structor  acceptability  of  the  TEDS  method  was  met  in  the 
SYS  223  courses,  the  following  decision  rule  was  set:  If 
greater  than  50  percent  of  instructor  responses  are  in  agree¬ 
ment  (disagreement)  with  questions  eight  and  nine  on  the 
instructor  questionnaire,  accept  the  null  hypothesis  that 
instructors  consider  TEDS  an  acceptable  (unacceptable)  deli¬ 
very  system. 

Assumptions 

In  any  statistical  research,  it  is  essential  to  clearly 

define  the  assumptions  that  had  to  be  made  about  the  probability 
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distributions  of  the  data  under  observation.  As  described 
earlier,  the  populations  included  all  AFSC  personnel  who  were 
eligible  for  enrollment  in  the  1980  offerings  of  SYS  223.  A 
second  population  included  AFIT  instructors  and  guest  lecturers 
who  qualify  to  lecture  in  SYS  223  courses.  These  assumptions 
include : 

1.  The  samples  were  randomly  selected  and  were 
independently  drawn. 

2.  The  means  used  in  ANOVA  analyses  are  normally 
distributed  from  populations  with  equal  vari- 


CHAPTER  III 


ANALYSIS  AND  INTERPRETATION 

This  chapter  summarizes  the  analysis  of  data  result¬ 
ing  from  an  application  of  the  methodology  described  in 
Chapter  II.  This  analysis  addressed  the  following  five  areas 

1.  Demographic  Data 

2.  End-of -Course  Questions  Data 

3.  Student  Acceptability  of  TEDS 

4.  Test  Results 

5.  Instructor  Acceptability  of  TEDS 

The  procedure  followed  in  this  chapter  was  to  exa¬ 
mine  each  area  above  in  turn  by  addressing  sequentially  the 
seven  combinations  listed  previously  in  Table  1.  Except  for 
end-of -course  questions,  analysis  results  were  not  discussed 
when  no  significant  differences  were  found.  Significance 
levels  less  than  the  five  percent  level  were  marked  by  an 
asterisk  on  those  tables  that  also  include  insignificant 
values.  First,  significant  demographic  variables  identified 
by  Chi-square  contingency  table  analysis  were  evaluated. 
Second,  significant  difference  in  end-of -course  questions 
responses  similarly  identified  were  evaluated.  Next,  signi¬ 
ficant  differences  in  student  acceptance  and  test  result 
scores,  computed  by  ANOVA  were  discussed.  Then  an  attempt 
was  made  to  identify  any  statistically  signi-ficant  predictor 
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variables  for  student  acceptance  and  test  scores  through 
regression  analysis.  Finally,  instructor  acceptance  was 
examined . 

Comparison  of  Demographic  Data 

The  percentage  distributions  of  the  demographic 
variables  (rank,  education  level,  age,  and  experience)  were 
compared  across  the  various  combinations  of  the  experimental 
design  using  crosstabulation .  The  contingency  table  results 
were  then  evaluated  using  the  Chi-square  statistic  to  deter¬ 
mine  if  the  student  groups  were  homogeneous  with  respect  to 
the  variable.  This  hypothesis  was  accepted  if  the  95  percent 
confidence  level  was  not  exceeded.  Therefore,  only  contin¬ 
gency  tables  that  were  less  than  five  percent  level  of  signi¬ 
ficance  are  depicted  for  demographic  variables. 

Comparison  of  End-of -Course  Data 

The  percentage  distributions  of  the  responses  to 
selected  end-of -course  questions  are  shown  using  the  Chi- 
square  contingency  table  analysis  described  in  Chapter  II. 
Contingency  table  values  exceeding  (less  than)  the  five  per¬ 
cent  significance  level  provided  evidence  to  reject  the 
hypothesis  that  the  variables  were  independent. 

Comparison  of  Student  Acceptance 
Bata  ‘ 

The  student  acceptance  of  the  TEDS  classes  was 
measured  by  combining  end-of -course  questionnaire  items  23 
and  25,  then  labeling  this  variable  acceptance.  For  ease  of 
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analysis,  the  combined  responses  were  coded  on  a  scale  from 
0  (high  acceptance  or  agreement)  to  8  (low  acceptance  or  dis¬ 
agreement).  Next,  mean  acceptance  values  were  computed  and 
compared  by  one-way  ANOVA  to  discover  any  significant  differ¬ 
ences  across  applicable  combinations  (that  is,  those  combina¬ 
tions  which  exclude  nonTEDS)  of  student  groups.  Finally, 
this  is  followed  by  stepwise  multiple  regression  to  analyze 
demographic  variables  and  location  in  terms  of  their  predictor 
value  for  explaining  significant  differences  discovered  by  the 
ANOVA  tests  on  acceptance.  The  variables  considered  for  pre¬ 
dictors  are: 

1.  Age  (Demographic  6) 

2.  Experience  (Demographic  7) 

3.  Grade  (Demographic  2) 

4.  Education  Level  (Demographic  3  or  4) 

5.  Location 

ESD 

SD 

HQ 

AD 

WPAFB  (TEDS  Resident) 

WPNOTT  (WPAFB  nonTEDS) 

Comparison  of  Test  Score  Data 

This  section  presents  the  analysis  used  to  discover 
significant  differences  between  the  learning  of  the  student 
groups  and  to  discover  any  predictor  variables  for  these  dif¬ 
ferences.  Test  scores  were  used  to  measure  this  learning 
variable.  These  test  score  variables  are:  pre-test  scores, 
post-test  scores,  and  achievement.  The  achievement  variable 
was  defined  in  Chapter  II  as  the  difference  between  post-tests 
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and  pre-tests. 

First,  significant  results  of  a  one-way  ANOVA  of  the 
mean  test  score  variable  are  presented.  This  is  followed  by 
a  stepwise  multiple  regression  to  analyze  demographic  vari¬ 
ables,  pre-test  score,  and  location  in  terms  of  their  poten¬ 
tial  predictor  value  for  explaining  significant  differences 
discovered  by  ANOVA  tests  on  post-test  scores  and  achievement. 
These  variables  are: 

1.  Age  (Demographic  6) 

2.  Experience  (Demographic  7) 

3.  Grade  (Demographic  2) 

4.  Education  Level  (Demographics  3  or  4) 

5.  Acceptance  (End-of -Course  23  and  25;  used  for 

combinations  5,  6  and  7  only) 

6.  Pre-test  Scores 

7.  Location  ( an  indicator  variable) 

ESD 

SD 

AD 

HQ 

WPAFB 

WPNOTT  (WPAFB  nonTEDS) 

Comparison  of  Instructor  Acceptance 
Data 

The  table  presenting  the  instructor  questionnaire 
(Appendix  C)  responses  was  constructed  similarly  to  the  tables 
for  end-of -course  questionnaire  responses.  For  question  11, 

"A  great  deal"  responses  are  listed  under  agree,  "Some"  res¬ 
ponses  are  listed  under  undecided,  and  "Not  at  all"  responses 
are  listed  under  disagree. 

The  percentage  of  responses  to  all  questions  except 
two  are  presented.  Questions  13  and  14  were  omitted  because 
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they  have  no  relevance  to  the  study.  However,  only  questions 
1-4,  6,  8-11,  and  15  are  discussed  since  they  are  directly 
related  to  TEDS  acceptance  by  instructors.  Due  to  circum¬ 
stances  beyond  the  researcher's  control,  there  was  a  six- 
month  delay  between  the  time  the  SYS  223  ended  and  when  the 
questionnaires  were  sent  out. 

Analysis  Results 

Demographic  Analysis  Results.  When  the  distributions 
of  rank/grade,  education  level,  age,  and  experience  were  com¬ 
pared  among  the  seven  combinations,  the  only  statistically 
significant  difference  revealed  was  between  the  nonTEDS  group 
and  the  TEDS  DT  group.  The  grade/rank  distributions  of  these 
two  groups  differed  (Table  3).  As  the  table  reveals,  the 
TEDS  DT  class  had  a  large  concentration  of  students  in  the 
lowest  grade  level,  while  the  nonTEDS  group  had  a  large  con¬ 
centration  of  students  in  the  middle  grade  level. 

End-of -Course  Questionnaire  Analysis  Results.  The 
tables  presenting  the  end-of -course  questionnaire  responses 
were  constructed  in  the  following  manner.  For  all  end-of- 
course  questions,  the  percentage  of  "Strongly  agree"  and 
"Agree"  responses  were  combined  and  tested  under  the  single 
heading  "Agree".  Similarly,  the  percentage  of  "Strongly  dis¬ 
agree"  and  "Disagree"  responses  were  combined  and  listed  under 
"Disagree". 

All  questions  except  13,  14,  16,  and  19  were  evalu¬ 
ated.  These  questions  were  not  germane  to  this  research.  In 
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TABLE  3 

CROSSTABULATION  OF  RANK  DISTRIBUTION 


Rank/Grade 

nonTEDS  % 

TEDS  DT  % 

Total 

E4 ,  01,  02 

GS5-GS10 

19 

40 

31 

E  5  ,  03,  GS11 

26 

17 

21 

E6 ,  04,  GS12 

43 

21 

30 

E7 ,  05,  GS13 

6 

14 

11 

E8  ,  E9 ,  06,  GS14 

6 

8 

7 

(Significance  Level:  .0367*) 
(Missing  Observations:  2.6%) 

*Signif icance  <  .05 


evaluating  the  responses,  greater  than  50  percent  agreement 
(disagreement)  was  interpreted  as  a  positive  (negative)  student 
attitude.  An  "R"  following  a  TEDS  class  represents  a  resident 
WPAFB  class.  In  addition,  "SL"  is  used  to  denote  the  signi¬ 
ficance  level  and  the  percent  of  missing  observations  is 
denoted  by  "MO".  In  addition  to  the  30  multiple  choice 
questions,  students  were  also  asked  for  their  written  res¬ 
ponses  to  the  questions  in  Part  II  of  their  questionnaires 
(Appendix  B) .  These  written  comments  are  included  in  the 
analysis  where  they  offer  support  to  the  questions  in  Part  I. 

Special  attention  should  be  given  to  the  percent  of 


missing  observations  as  the  potential  exists  for  the  occasion¬ 
ally  large  numbers  of  missing  data  to  skew  the  analysis  re¬ 
sults.  For  example,  due  to  circumstances  beyond  the 


researcher’s  control,  the  entire  SYS  223  TEDS  CT  Armament  Divi 
sion’s  end-of -course  questionnaire  responses  are  missing  from 
the  data  base  (18  students) . 

Before  examining  the  questions,  it  is  necessary  to 
point  out  any  differences  that  may  have  had  an  effect  on  the 
comparisons.  First,  Systems  Program  Management  had  been  re¬ 
organized  with  exercises  introduced  for  the  first  time  in  the 
nonTEDS  offering.  Second,  the  TEDS  CT  offering  was  the  first 
exposure  to  the  TEDS  delivery  method  by  the  course  instructor 
and  most  guest  speakers.  Third,  the  nonTEDS  schedule  was  six 
hours  of  class  per  day,  while  the  TEDS  classes  were  conducted 
for  four  hours  per  day.  Finally,  the  difference  in  class  day 
allowed  the  nonTEDS  class  to  cover  the  material  faster.  As 
one  result,  they  only  had  two  longer  tests,  whereas  the  two 
TEDS  offerings  had  three  tests  that  constituted  their  post¬ 
test  score  (8)  . 

Responses  to  Question  1,  "The  course  objectives  were 
made  clear  either  orally  or  in  the  instructional  aids,”  are 
displayed  in  Table  4.  The  only  significant  difference  indi¬ 
cated  in  Table  4  was  between  the  nonTEDS  and  TEDS  CT  classes. 
However,  all  groups  appear  to  agree  that  the  course  objectives 
were  clear. 

Responses  to  Question  2,  "The  course  appeared  well 
structured,"  are  displayed  in  Table  5.  Analysis  of  Question 
2  responses  indicated  significant  differences  with  two  of  the 
combinations.  The  TEDS  DT  resident  class  appeared  to  have  a 
much  higher  percentage  of  agreement  than  either  the  nonTEDS 
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TABLE  4 


QUESTION  1:  OBJECTIVES  MADE  CLEAR 


Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT 

70.9 

14.5 

14.5 

nonTEDS 

90.9 

4.5 

4.5 

(SL  = 

.0482* 

MO  =  271) 

TEDS  DT 

82.1 

12.5 

5.4 

nonTEDS 

90.9 

4.5 

4.5 

(SL  = 

.3707 

MO  =  131) 

TEDS  CT(R) 

85.0 

5.0 

10.0 

nonTEDS 

90.9 

4.5 

4.5 

(SL  = 

.6996 

MO  =  11%) 

TEDS  DT  (R) 

95.0 

5.0 

0 

nonTEDS 

90.9 

4.5 

4.5 

(SL  = 

.6250 

f - N 

cr> 

ii 

O 

TEDS  CT  (R) 

85.0 

5.0 

10.0 

TEDS  DT  (R) 

95.0 

5.0 

0 

(SL  = 

.  3480 

MO  =  13%) 

TEDS  CT 

70.9 

14.5 

14.5 

TEDS  DT 

82.1 

12.5 

5.4 

(SL  = 

.  2337 

/■ — ' 

CO 

CM 

ft 

O 

T. 

Remotes  CT 

62.9 

20.0 

17 . 1 

Remotes  DT 

75.0 

16.7 

8.3 

(SL  = 

.4554 

MO  *  35%) 

*  Significance  <  .05 


TABLE  S 


OUESTION  2:  COURSE  WELL  STRUCTURED 


Class 

Agree  % 

Undecided 

% 

Disagree  % 

TEDS  CT 

50.9 

18.2 

30.9 

nonTEDS 

63.6 

20.5 

15.9 

(SL  = 

.2193 

MO  = 

38%) 

TEDS  DT 

71.4 

12.5 

16.1 

nonTEDS 

63.6 

20.5 

15.9 

(SL  = 

.  5502 

MO  = 

13%) 

TEDS  CT  (R) 

55.0 

35.0 

10.0 

nonTEDS 

63.6 

20.5 

15.9 

(SL  = 

.4331 

MO  - 

11%) 

TEDS  DT  (R) 

95.0 

0 

5.0 

nonTEDS 

63.6 

20.5 

15.9 

(SL  = 

.0268* 

MO  = 

9%) 

TEDS  CT  (R) 

55.0 

35.0 

10.0 

TEDS  DT  (R) 

95.0 

0 

5.0 

(SL  = 

.0088* 

MO  = 

13%) 

TEDS  CT 

50.9 

18.2 

30.9 

TEDS  DT 

71.9 

12.5 

16.1 

(SL  = 

.0781 

MO  => 

28%) 

Remotes  CT 

48.6 

8.6 

42.9 

Remotes  DT 

58.3 

19.4 

22.2 

(SL  - 

.1263 

MO  - 

35%) 

*Signif icance  < 

.05 

1 


class  or  the  TEDS  CT  resident  class.  However,  all  classes 
except  the  combined  remotes  of  TEDS  CT  showed  greater  than  50 
percent  agreeing  that  the  course  was  well  structured.  Even 
the  TEDS  CT  remotes  had  a  slightly  higher  percent  of  agree¬ 
ment  than  disagreement. 

Further  examination  of  written  comments  of  the  stu¬ 
dents  showed  that  many  students,  mainly  in  TEDS  CT ,  and 
particularly  in  the  resident  portion  of  TEDS  CT,  felt  the 
course  was  somewhat  disorganized. 

Responses  to  Question  3,  "The  course  structure  per¬ 
mitted  questions  to  be  asked  and  answered  satisfactorily," 
are  displayed  in  Table  6.  Analysis  of  Question  3  responses 
indicated  that  all  groups  agreed  that  the  course  structure 
permitted  questions.  However,  significant  differences  showed 
up  in  four  of  the  seven  combinations.  The  nonTEDS  and  the 
three  breakdowns  of  TEDS  DT  (TEDS  DT,  TEDS  DT  resident  and 
TEDS  DT  remotes)  all  had  very  strong  agreement.  Apparently, 
TEDS  CT,  especially  the  resident  class  of  TEDS  CT,  was  less 
satisfied  than  the  other  groups. 

Responses  to  Question  4,  "The  room  was  conducive  to 
learning,"  are  displayed  in  Table  7.  Analysis  of  Question  4 
responses  indicated  that  all  classes  except  TEDS  CT  resident 
were  convinced  that  the  room  was  conducive  to  learning.  This 
is  interesting  since  the  resident  classes  of  TEDS  CT  and 
TEDS  DT  were  both  held  in  room  112  of  the  AFIT’s  School  of 
Systems  and  Logistics.  The  large  nonTEDS  class  was  held  in 
rooms  320  and  322  combined.  Significant  differences  were 
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TABLE  6 


QUESTION  3:  STRUCTURE  PERMITTED  QUESTIONS 


Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT 

65.5 

12.7 

21.8 

nonTEDS 

90.9 

6.8 

2.3 

(SL  = 

.0067* 

MO  = 

27%) 

TEDS  DT 

85.7 

5.4 

8.9 

nonTEDS 

90.9 

6.8 

2.3 

(SL  = 

.3711 

MO  = 

13%) 

TEDS  CT  (R) 

55.0 

10.0 

35.0 

nonTEDS 

90.9 

6.8 

2.3 

(SL  = 

.0008* 

MO  = 

11%) 

TEDS  DT  (R) 

95.0 

0 

5.0 

nonTEDS 

90.9 

6.8 

2.3 

(SL  = 

.4240 

MO  = 

9%) 

TEDS  CT  (R) 

55.0 

10.0 

35.0 

TEDS  DT  (R) 

95.0 

0 

5.0 

(SL  = 

.0133* 

MO  = 

13%) 

TEDS  CT 

65.5 

12.7 

21.8 

TEDS  DT 

85.7 

5.4 

8.9 

(SL  - 

.0453* 

MO  = 

28%) 

Remotes  CT 

71.4 

14.3 

14.3 

Remotes  DT 

80.6 

8.3 

11.1 

(SL  * 

.6397 

MO  = 

35%) 

*  Significance  < 

.OS 

TABLE  7 


QUESTION  4:  ROOM  CONDUCIVE  TO  LEARNING 


Class 

Agree  % 

Undecided 

O. 

0 

Disagree  % 

TEDS  CT 

58.2 

16.4 

25.5 

nonTEDS 

84.1 

13.6 

2.3 

(SL  = 

. 0038* 

MO  = 

27%) 

TEDS  DT 

67.9 

16.1 

16.1 

nonTEDS 

84.1 

13.6 

2.3 

(SL  = 

.  0592 

MO  = 

13%) 

TEDS  CT  (R) 

45.0 

35.0 

20.0 

nonTEDS 

84.1 

15.6 

2.3 

(SL  = 

.0031* 

MO  «= 

11%) 

TEDS  DT  (R) 

80.0 

15.0 

5.0 

nonTEDS 

84.1 

13.6 

2.3 

(SL  = 

.  8298 

MO  = 

9%) 

TEDS  CT  (R) 

45.0 

35.0 

20.0 

TEDS  DT  (R) 

80.0 

15.0 

5.0 

(SL  = 

.  0686 

MO  = 

13%) 

TEDS  CT 

S8.2 

16.4 

25.5 

TEDS  DT 

67.9 

16.1 

16.1 

(SL  = 

.4510 

MO  = 

28%) 

Remotes  CT 

65.7 

5.7 

28.6 

Remotes  DT 

61.1 

16.7 

9  9  9 

i*  •  *0 

(SL  * 

.  32"8 

MO  * 

35%) 

*  Significance  < 

.  0  5 

\ 
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found  between  TEDS  CT  and  nonTEDS  and  between  resident  CT 
and  nonTEDS. 

Responses  to  Question  5,  "I  was  in  a  position  where 
I  could  hear  and  see  well,"  are  displayed  in  Table  8.  Analy¬ 
sis  of  Question  5  responses  indicated  no  significant  differ¬ 
ence  among  the  classes.  All  classes  agreed  that  they  could 
hear  and  see  well. 

Responses  to  Question  6,  "There  should  have  been 
more  handout  materials,"  are  displayed  in  Table  9.  Analysis 
indicated  that  there  were  significant  differences  between 
TEDS  DT  and  nonTEDS,  between  resident  TEDS  DT  and  nonTEDS, 
between  TEDS  CT  and  TEDS  DT ,  and  between  the  remotes  of  TEDS 
CT  and  TEDS  DT.  Overall,  only  resident  TEDS  CT  and  remotes 
TEDS  DT  felt  that  there  should  not  be  more  handouts.  The  re¬ 
maining  classes  had  less  than  a  50  percent  response  to  any 
answer.  It  should  be  noted  that  this  was  probably  a  distri¬ 
bution  problem  which  was  corrected  for  TEDS  DT  (8) . 

Responses  to  Question  7,  "The  course  should  have  been 
longer,"  are  displayed  in  Table  10.  Analysis  of  Question  7 
responses  indicated  no  significant  differences  among  the 
classes.  All  classes  disagreed  that  the  course  should  be 
longer . 

Responses  to  Question  8,  "My  time  could  have  been 
better  utilized  elsewhere,"  are  displayed  in  Table  11. 

Analysis  of  Question  8  responses  indicated  that  the  only  sig¬ 
nificant  difference  was  between  the  resident  classes  of  TEDS 
CT  and  TEDS  DT.  However,  both  of  these  classes  disagreed 
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TABLE  8 


QUESTION  5:  I  COULD  SEE  AND  HEAR  WELL 
Class  Agree  %  Undecided  %  Disagree  % 


TEDS  CT 

89.1 

5.5 

5.5 

nonTEDS 

88.6 

6.8 

4.5 

(SL  =  .9439 

MO 

= 

271) 

TEDS  DT 

92.9 

3 . 6 

3.6 

nonTEDS 

88.6 

6.8 

4.5 

(SL  =  .7312 

MO 

= 

13%) 

TEDS  CT 

(R) 

100.0 

0 

0 

nonTEDS 

88.6 

6.8 

4.5 

(SL  =  .2915 

MO 

= 

11%) 

TEDS  DT 

(R) 

100.0 

0 

0 

nonTEDS 

88.6 

6.8 

4.5 

(SL  =■=  .2915 

MO 

= 

9%) 

TEDS  CT 

(R) 

100.0 

0 

0 

TEDS  DT 

(R) 

100.0 

0 

0 

(SL  =  1.0 

MO 

s 

13%) 

TEDS  CT 

89.1 

5.5 

5.5 

TEDS  DT 

92.9 

3.6 

3.6 

(SL  =■  .  7866 

MO 

= 

28%) 

Remotes 

CT 

82.9 

8.6 

8.6 

Remotes 

DT 

88.9 

5.6 

5.6 

(SL  -  .7658 

MO 

3 

35%) 
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TABLE  9 


QUESTION  6:  SHOULD  BE  MORE  HANDOUTS 


Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT 

29.1 

29.1 

41.8 

nonTEDS 

25.0 

29.5 

45.5 

(SL  = 

.  8931 

MO  = 

27%) 

TEDS  DT 

5.4 

48 . 2 

46.4 

nonTEDS 

25.0 

29.5 

45.5 

(SL  = 

.0114* 

MO  = 

13%) 

TEDS  CT  (R) 

10.0 

30.0 

60.0 

nonTEDS 

25.0 

29.5 

45.5 

(SL  = 

.  3489 

MO  = 

11%) 

TEDS  DT  (R) 

5.0 

60.0 

35.0 

nonTEDS 

25.0 

29.5 

45.5 

(SL  = 

.0377* 

MO  = 

9%) 

TEDS  CT  (R) 

10.0 

30.0 

60.0 

TEDS  DT  (R) 

5.0 

60.0 

35.0 

(SL  * 

.  1613 

MO  = 

13%) 

TEDS  CT 

29.1 

29.1 

41.8 

TEDS  DT 

5.4 

48.2 

46.4 

(SL  - 

.0026* 

MO  = 

28%) 

Remotes  CT 

40.0 

28.6 

31.4 

Remotes  DT 

5.6 

41.7 

52.8 

(SL  - 

.0023* 

MO  * 

35%) 

"  Significance  < 
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TABLE  10 


QUESTION  7:  COURSE  SHOULD  BE  LONGER 


Class  Agree  %  Undecided  %  1 


TEDS  CT 

5.5 

7.3 

87.3 

nonTEDS 

11.4 

11.4 

77.3 

(SL  = 

.4063 

MO 

= 

271) 

TEDS  DT 

12.5 

16.1 

71.4 

nonTEDS 

11.4 

11.4 

77.3 

(SL  = 

.7671 

MO 

= 

13%) 

TEDS  CT 

(R) 

5.0 

15.0 

80.0 

nonTEDS 

11.4 

11.4 

77.3 

(SL  = 

.6864 

MO 

= 

11%) 

TEDS  DT 

(R) 

20.0 

15.0 

65.0 

nonTEDS 

11.4 

11.4 

77.3 

(SL  = 

.  5608 

MO 

= 

9%) 

TEDS  CT 

(R) 

5.0 

15.0 

80.0 

TEDS  DT 

(R) 

20.0 

15.0 

65.0 

(SL  = 

.3481 

MO 

= 

13%) 

TEDS  CT 

5 .  5 

7.3 

87.3 

TEDS  DT 

12.5 

16.1 

71.4 

(SL  =* 

.1199 

MO 

= 

28%) 

Remotes 

CT 

5.7 

2.9 

91.4 

Remotes 

DT 

8.3 

16.7 

75.0 

(SL  - 

.1236 

MO 

X 

35%) 

TABLE  11 


QUESTION  8:  TIME  BETTER  UTILIZED  ELSEWHERE 


Class  Agree  %  Undecided  %  Disagree 


TEDS  CT 

29.1 

29.1 

41.8 

nonTEDS 

13.6 

22.7 

63.6 

(SL  = 

.0720 

MO 

= 

27%) 

TEDS  DT 

14.5 

25  .  5 

60.0 

nonTEDS 

13.6 

22.7 

63.6 

(SL  =■ 

.9316 

MO 

= 

14%) 

TEDS  CT 

(R) 

?0.0 

30.0 

50.0 

nonTEDS 

13.6 

22.7 

63.6 

(SL 

.  5834 

MO 

= 

11%) 

TEDS  DT 

(R) 

0 

15.0 

85.0 

nonTEDS 

13.6 

22.7 

63.6 

(SL  * 

.1337 

MO 

S 

9%) 

TEDS  CT 

(R) 

20.0 

30.0 

50.  0 

TEDS  DT 

(R) 

0 

15.0 

85.0 

(SL  = 

.0331* 

MO 

a 

13%) 

TEDS  CT 

29.1 

29.1 

41.8 

TEDS  DT 

14.5 

25.5 

60.0 

(SL  = 

.1010 

MO 

= 

29%) 

Remotes 

CT 

34.3 

28.6 

37.1 

Remotes 

DT 

22.9 

31.4 

45.7 

(SL  * 

.  5605 

MO 

a 

36%) 

*  Significance  <  .05 
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that  their  time  could  be  better  utilized  elsewhere.  The 
different  groups  of  TEDS  CT  had  SO  percent  or  less  to  dis¬ 
agree.  Interestingly,  TEDS  DT  remotes  also  had  less  than  50 
percent  to  disagree.  Therefore,  both  remote  groups  were 
divided  among  the  three  choices. 

Responses  to  Question  9,  "I  will  be  able  to  do  my 
job  better  as  a  result  of  this  course,"  are  displayed  in 
Table  12.  Analysis  to  Question  9  responses  indicated  no 
significant  differences  among  the  classes.  All  classes  agreed 
that  they  would  be  able  to  do  a  better  job  as  a  result  of  the 
course . 

Responses  to  Question  10,  "The  course  met  my  expec¬ 
tations,"  are  displayed  in  Table  13.  Analysis  of  Question  10 
responses  indicated  significant  differences  with  five  of  the 
combinations  as  shown  in  the  table.  Differences  were  found 
in  all  combinations  except  nonTEDS  to  TEDS  DT  and  the  remotes 
of  TEDS  CT  to  the  remotes  of  TEDS  DT.  NonTEDS,  TEDS  DT  and 
TEDS  DT  resident  agreed  that  the  course  met  their  expectations. 
The  remaining  classes  (including  the  remotes  of  TEDS  DT)  did 
not  have  50  percent  responding  to  any  choice. 

Responses  to  Question  11,  "The  course  was  more  infor¬ 
mative  than  I  had  anticipated,"  are  displayed  in  Table  14. 
Analysis  of  Question  11  responses  indicated  a  significant 
difference  only  between  TEDS  CT  and  nonTEDS.  In  general, 
greater  than  50  percent  of  the  TEDS  CT  breakdowns  were  un¬ 
decided  across  the  combinations;  whereas,  the  other  groups 
were  divided  among  the  three  choices. 


TABLE  12 


QUESTION  9:  WILL  DO  JOB  BETTER 
Class  Agree  %  Undecided  % 


TEDS  CT 

69.1 

23.6 

nonTEDS 

86.4 

13.6 

(SL  = 

.0664 

MO 

= 

35%) 

TEDS  DT 

80.4 

16.1 

nonTEDS 

86.4 

13.6 

(SL  = 

.4115 

MO 

= 

13%) 

TEDS  CT 

(R) 

80.0 

15.0 

nonTEDS 

86.4 

13.6 

(SL  = 

.3192 

MO 

= 

11%) 

TEDS  DT 

(R) 

100.0 

0 

nonTEDS 

86.4 

13.6 

(SL  = 

.2033 

MO 

= 

9%) 

TEDS  CT 

(R) 

80.0 

15.0 

TEDS  DT 

(R) 

100.0 

0 

(SL  = 

.  1084 

MO 

a 

13%) 

TEDS  CT 

69.1 

23.6 

TEDS  DT 

80.4 

16.1 

(SL  = 

.  3724 

MO 

a 

28%) 

Remotes 

CT 

62.9 

28.6 

Remotes 

DT 

69.4 

25.0 

(SL  - 

.8065 

MO 

a 

35%) 

TABLE  13 


QUESTION  10:  COURSE  MET  EXPECTATIONS 


Class 

Agree 

1 

Undecided  % 

Disagree  % 

TEDS  CT 

23.6 

40.0 

36.4 

nonTEDS 

56.8 

20.5 

22.7 

(SL  = 

.0032* 

MO  = 

27%) 

TEDS  DT 

58.9 

19.6 

21.4 

nonTEDS 

S6.8 

20.5 

22.7 

(SL  - 

.9773 

MO  = 

13%) 

TEDS  CT 

(R) 

25.0 

50.0 

25.0 

nonTEDS 

56 . 5 

20.5 

22.7 

(SL  = 

.0296* 

MO  = 

11%) 

TEDS  DT 

(R) 

100.0 

0 

0 

nonTEDS 

56.8 

20.5 

22.7 

(SL  = 

.0022* 

MO  = 

9%) 

TEDS  CT 

(R) 

25.0 

50.0 

25.0 

TEDS  DT 

(R) 

100.0 

0 

0 

(SL  = 

.0000* 

MO  » 

13%) 

TEDS  CT 

23.6 

40.0 

36.4 

TEDS  DT 

58.9 

19.6 

21.4 

(SL  * 

.0008* 

MO  - 

28%) 

Remotes 

CT 

22.9 

34.3 

42.9 

Remotes 

DT 

36.1 

30.6 

33.3 

(SL  * 

.  4599 

MO  - 

35%) 

* 


Significance  <  .05 


TABLE  14 


QUESTION  11: 

COURSE  MORE  IMFORMATIVE 

THAN 

ANTICIPATED 

Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT 

12.7 

50.9 

36.4 

nonTEDS 

34.1 

38.6 

27.3 

(SL  = 

.0396* 

MO= 

27%) 

TEDS  DT 

30.4 

39.3 

30.4 

nonTEDS 

34.1 

38.6 

27.3 

(SL  = 

.9090 

MO  = 

13%) 

TEDS  CT  (R) 

15.0 

50.0 

35.0 

nonTEDS 

34.1 

38.6 

27.3 

(SL  = 

.2895 

MO  = 

11%) 

TEDS  DT  (R) 

45.0 

40.0 

15.0 

nonTEDS 

34.1 

38.6 

27.3 

(SL  = 

.  5152 

MO  = 

9%) 

TEDS  CT  (R) 

15.0 

50.0 

35.0 

TEDS  DT  (R) 

45.0 

40.0 

15.0 

(SL  * 

.0897 

MO  = 

13%) 

TEDS  CT 

12.7 

50.9 

36.4 

TEDS  DT 

30.4 

39.3 

30.4 

(SL  * 

.  0773 

MO  =* 

28%) 

Remotes  CT 

11.4 

51.4 

37.1 

Remotes  DT 

22.2 

38.9 

38.9 

(SL  - 

.  3952 

MO  * 

35% 

j  *  Significance  <  .05 


Responses  to  Question  12,  "Overall,  the  course  was 
extremely  difficult,"  are  displayed  in  Table  15.  Analysis  of 
Question  12  responses  indicated  no  significant  differences. 
Overall,  all  groups  did  not  think  the  course  extremely  diffi¬ 
cult.  Only  the  TEDS  CT  resident  class  had  less  than  50  per¬ 
cent  disagree  with  the  question. 

Responses  to  Question  15,  "Discussion  of  the  tests 
helped  me  learn,"  are  displayed  in  Table  16.  Analysis  of 
Question  15  responses  indicated  significant  differences 
with  all  combinations  except  nonTEDS  to  resident  TEDS  CT , 
resident  TEDS  CT  to  resident  TEDS  DT,  and  TEDS  CT  to  TEDS  DT. 
The  TEDS  classes  generally  disagreed  with  the  question.  The 
written  comments  suggested  that  the  time  lapse  between  the 
tests  and  the  discussions  of  the  tests  was  too  long. 

Responses  to  Question  17,  "I  liked  the  hours  the 
course  was  offered,"  are  displayed  in  Table  16.  Analysis  of 
Question  17  responses  indicated  significant  differences  with 
the  first  four  combinations.  TEDS  CT,  resident  TEDS  CT ,  and 
the  remotes  of  TEDS  DT  groups  clearly  disliked  the  hours. 

TEDS  DT  and  the  remotes  of  TEDS  CT  had  more  students  who  dis¬ 
liked  the  hours  than  those  who  favored  the  hours.  Obviously, 
the  nonTEDS  group,  which  was  overwhelmingly  pleased  with  the 
hours,  accounted  for  the  differences. 

Time  of  day  differed  for  the  groups  depending  upon 
the  time  zones  in  which  the  class  was  located.  TEDS  classes 
were  conducted  from  nine  o'clock  in  the  morning  to  one  o'clock 
in  the  afternoon  for  the  Pacific  Time  Zone  students  (SD  only), 
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TABLE  15 


QUESTION  12:  OVERALL  COURSF  EXTREMELY  DIFFICULT 


Class 

Agree 

% 

Undecided  % 

Disagree  % 

TEDS  CT 

9.1 

27.3 

63.6 

nonTEDS 

9.1 

27.3 

63.6 

(SL 

= 

1.000 

MO 

= 

27%) 

TEDS  DT 

16.1 

32.1 

51.8 

nonTEDS 

9.1 

27.3 

63.6 

(SL 

= 

.4220 

MO 

= 

13%) 

TEDS  CT 

(R) 

5.0 

25.0 

70.0 

nonTEDS 

9.1 

27.3 

63.6 

(SL 

= 

.8162 

MO 

a 

11%) 

TEDS  DT 

(R) 

25.0 

35.0 

40.0 

nonTEDS 

9.1 

27.3 

63.6 

(SL 

= 

.1276 

MO 

= 

9%) 

TEDS  CT 

(R) 

5.0 

25.0 

70.0 

TEDS  DT 

(R) 

25.0 

35.0 

40.0 

(SL 

= 

.0985 

MO 

= 

13%) 

TEDS  CT 

9.1 

27.3 

63.6 

TEDS  DT 

16.1 

32.1 

51.8 

(SL 

3 

.  3736 

MO 

3 

28%) 

Remotes 

CT 

11.4 

28.6 

60.0 

Remotes 

DT 

11.1 

30.6 

58.3 

(SL 

3 

.9834 

MO 

3 

35%) 
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TABLE  16 


QUESTION  15:  TEST  DISCUSSION  HELPED  ME  LEARN 
Class  Agree  %  Undecided  %  Disagree  % 


TEDS  CT 

36.4 

10.9 

52.7 

nonTEDS 

56.8 

15.9 

27.3 

(SL  = 

.0380*  MO 

= 

21%') 

TEDS  DT 

30.4 

28.6 

41.1 

nonTEDS 

56.8 

15.9 

27.3 

(SL  - 

.0278*  MO 

= 

13%) 

TEDS  CT 

no 

35.0 

15.0 

50.0 

nonTEDS 

56.8 

15.9 

27 . 3 

(SL  = 

.1843  MO 

= 

11%) 

TEDS  DT 

no 

30.0 

10.0 

60.0 

nonTEDS 

56.8 

15.9 

27.3 

(SL  = 

.0426*  MO 

s 

9%) 

TEDS  CT 

00 

35.0 

15.0 

50.0 

TEDS  DT 

00 

30.0 

10.0 

60.0 

(SL  - 

.7950  MO 

= 

13%) 

TEDS  CT 

36.4 

10.9 

52.7 

TEDS  DT 

30.4 

28.6 

41.1 

(SL  * 

.0648  MO 

s 

28%) 

Remotes 

CT 

37.1 

8.6 

54.3 

Remotes 

DT 

30.6 

38.9 

30.6 

(SL  = 

.0091*  MO 

3 

35%) 

*  Significance  <  .05 
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TABLE  17 


QUESTION  17:  LIKED  HOURS  COURSE  OFFERED 


Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT 

32.7 

12.7 

54.5 

nonTEDS 

95.5 

0 

4.5 

(SL  = 

.0000* 

MO 

= 

27%) 

TEDS  DT 

39.3 

14.3 

46.4 

nonTEDS 

95.5 

0 

4.5 

(SL  = 

.0000* 

MO 

= 

13%) 

TEDS  CT  (R) 

30.0 

5.0 

65.0 

nonTEDS 

95.5 

0 

4.5 

(SL  = 

.0000* 

MO 

11%) 

TEDS  DT  (R) 

40.0 

25.0 

35.0 

nonTEDS 

95.5 

0 

4.5 

(SL  = 

.0000* 

MO 

= 

9%) 

TEDS  CT  (R) 

30.0 

5.0 

65.0 

TEDS  DT  (R) 

40.0 

25.0 

35.0 

(SL  = 

.  0929 

MO 

= 

13%) 

TEDS  CT 

32.7 

12.7 

54.5 

TEDS  DT 

39.3 

14.3 

46.4 

(SL  * 

.6895 

MO 

= 

28%) 

Remotes  CT 

34.4 

17.1 

48.6 

Remotes  DT 

38.9 

8.3 

52.8 

(SL  = 

.  5350 

MO 

Z 

35% 

*  Significance  < 

.05 
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from  eleven  o'clock  in  the  morning  until  three  o'clock  in  the 
afternoon  for  Central  Time  Zone  students  (AD  only) ,  and  from 
noon  until  four  o'clock  in  the  afternoon  for  all  classes  in 
the  Eastern  Time  Zone  (ESD,  HQ,  and  WPAFB) .  The  necessity  to 
hold  classes  during  the  normal  duty  day  dictated  this  sche¬ 
dule.  The  nonTEDS  class  was  conducted  at  WPAFB  from  nine 
o'clock  in  the  morning  until  three  o'clock  in  the  afternoon 
with  a  one-hour  lunch  break. 

Responses  to  Question  18,  "I  learn  more  from  a  course 
when  I  am  TDY  (completely  removed  from  my  job  location),"  are 
displayed  in  Table  18.  Analysis  of  Question  18  responses 
indicated  that  all  classes  felt  that  they  learned  more  when 
they  were  TDY.  The  only  significant  difference  appeared 
between  the  TEDS  DT  and  the  nonTEDS.  Further  examination  of 
student  written  comments  revealed  that  TEDS  students  were 
required  to  work  on  their  normal  jobs  for  half  a  day  and 
attend  class  the  other  half.  They  seemed  to  feel  that  this 
took  away  from  their  ability  to  concentrate  on  the  Program 
Management  course. 

Responses  to  Question  20,  "The  'class  day'  should 
be:  A.  1-2  hours;  B.  2-3  hours;  C.  3-4  hours;  D.  4-5  hours; 

E.  5-6  hours,"  are  displayed  in  Table  19.  Analysis  of  Ques¬ 
tion  20  responses  indicated  that  there  were  significant  dif¬ 
ferences  for  each  comparison  which  included  the  nonTEDS 
group  (combination  1-4).  The  nonTEDS  group  favored  a  5-6  hour 
class  day  (which  they  had) .  The  overall  TEDS  CT  sections 
seemed  to  favor  a  3-4  hour  day  (which  they  had);  whereas, 
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TABLE  18 


QUESTION  18: 

I  LEARN  MORE  WHEN 

TDY 

Class 

Agree 

%  Undecided  % 

Disagree  % 

TEDS  CT 

78.2 

14.5 

7.3 

nonTEDS 

75.0 

22.7 

2.3 

(SL  - 

.  3426 

MO  =  27%) 

TEDS  DT 

67.9 

14.3 

17.9 

nonTEDS 

75.0 

22.7 

2.3 

(SL  = 

.0370* 

MO  =  13%) 

TEDS  CT 

(R) 

70.0 

20.0 

10.0 

nonTEDS 

75.0 

22.7 

2.3 

(SL  = 

.3975 

MO  =  11%) 

TEDS  DT 

(R) 

55.0 

30.0 

15.0 

nonTEDS 

75.0 

22.7 

2.3 

(SL  = 

.0976 

MO  *  9%) 

TEDS  CT 

(R) 

70.0 

20.0 

10.0 

TEDS  DT 

(R) 

55.0 

30.0 

15.0 

(SL  = 

.6188 

MO  =  13%) 

TEDS  CT 

78.2 

14.5 

7.3 

TEDS  DT 

67.9 

14.3 

17.9 

(SL  = 

.  2380 

MO  =  28%) 

Remotes 

CT 

82.9 

11.4 

5.7 

Remotes 

DT 

7  S .  0 

5.6 

19.4 

(SL  = 

.1736 

MO  =  35%) 

i 

*  Significance  <  .05 
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TABLE  19 


QUESTION  20:  CLASS  DAY  SHOULD  BE 


Class 

Hours 

1-2 

2-3 

3-4 

4-5 

5-6 

TEDS  CT 

5.5 

10.9 

45.5 

3.6 

34.5 

nonTEDS 

0 

0 

2.3 

20.5 

77.3 

(SL  = 

.0000* 

MO  = 

27%) 

TEDS  DT 

7.1 

12.5 

35.7 

8.9 

35.7 

nonTEDS 

0 

0 

2.3 

20.5 

77.3 

(SL  = 

.0000* 

MO  = 

13%) 

TEDS  CT 

(R) 

50.0 

5.0 

45.0 

-nonTEDS 

2.3 

20.5 

77.3 

(SL  = 

.0000* 

MO  = 

11%) 

TEDS  DT 

(R) 

40.0 

15.0 

45.0 

nonTEDS 

2.3 

20.5 

77.3 

(SL  - 

.0003* 

MO  = 

9%) 

TEDS  CT 

(R) 

50.0 

5.0 

45.0 

TEDS  DT 

(R) 

40.0 

15.0 

45.0 

(SL  = 

.  5427 

MO  = 

13%) 

TEDS  CT 

5.5 

10.9 

45.5 

3.6 

34.5 

TEDS  DT 

7.1 

12.5 

35.7 

8.9 

35.7 

(SL  = 

.7214 

MO  3 

28%) 

Remotes 

CT 

8.6 

17.1 

42.9 

2.9 

28.6 

Remotes 

DT 

11.1 

19.4 

33.3 

5.6 

30.6 

(SL  - 

.9217 

MO  = 

35%) 

*  Significance  < 

.05 
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overall  TEDS  DT  were  mainly  split  between  a  3-4  hour  day  and 
a  5-6  hour  day. 

Previous  analysis  of  Question  17  responses  indicated 
that  most  of  the  TEDS  combinations  found  fault  with  the  TEDS 
schedule,  but  student  responses  to  Question  20  did  not  indi¬ 
cate  that  they  were  greatly  displeased  with  the  length  of 
their  class  day.  Furthermore,  in  written  comments,  students 
complained  that  their  supervisors  expected  too  much  of  them 
on  their  normal  jobs  in  addition  to  attending  class.  This 
may  have  caused  their  dislike  of  the  TEDS  schedule.  The  fact 
that  the  nonTEDS  class,  which  consisted  of  students  TDY  at 
WPAFB  away  from  their  jobs,  liked  their  schedule  and  class 
hours  supported  this  theory. 

Responses  to  Question  23,  "The  Teleteach  delivery 
system  is  an  acceptable  learning  medium,"  are  displayed  in 
Table  20.  Analysis  of  Question  23  responses  indicated  that 
the  comparison  between  the  resident  classes  of  TEDS  CT  and 
TEDS  DT  was  the  only  one  to  differ  significantly.  In  that 
combination,  resident  TEDS  DT  agreed  that  Teleteach  was 
acceptable,  while  resident  TEDS  CT  did  not  agree.  Overall, 
the  delivery  system  was  apparently  more  acceptable  to  the 
TEDS  DT  group.  However,  this  difference  was  primarily  caused 
by  the  resident  class  responses.  To  repeat,  both  sets  of 
remote  classes  agreed  that  TEDS  was  acceptable  by  approxi¬ 
mately  56  percent  (see  discussion  at  the  end  of  this  section 
for  further  comments) . 
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TABLE  20 


QUESTION  23:  TELETEACH  IS  ACCEPTABLE 
LEARNING  MEDIUM 


Class 

Agree  % 

Undecided 

% 

Disagree  % 

TEDS  CT  (R) 

25.0 

20.0 

55.0 

TEDS  DT  (R) 

65.0 

35.0 

0 

(SL  = 

.0005* 

MO  = 

9 %) 

TEDS  CT 

45.5 

16.4 

38.2 

TEDS  DT 

58.9 

23.2 

17.9 

(SL  = 

.0571 

MO  = 

28?) 

Remotes  CT 

S7.1 

14.3 

28.6 

Remotes  DT 

SS.6 

16.7 

27.8 

(SL  - 

.9623 

MO  = 

3  5?) 

*  Significance  < 

.05 

Responses  to  Question  24,  "The  teleconferencing  equip¬ 
ment  (mikes  and  blackboard)  was  easy  to  operate,"  are  displayed 
in  Table  21.  Analysis  of  Question  24  responses  indicated  a 
significant  difference  between  the  TEDS  resident  classes. 
Overall,  all  groups  with  the  exception  of  the  TEDS  CT  resident 
class  agreed  that  the  teleconferencing  equipment  was  easy  to 
operate  (see  discussion  at  end  of  this  section  for  further 
comments) . 

Responses  to  Question  25,  ”1  would  take  another 
course  which  used  this  delivery  system,"  are  displayed  in 
Table  22.  Analysis  of  Question  25  responses  indicated  signi¬ 
ficant  differences  between  both  TEDS  CT  and  TEDS  DT ,  and 
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TABLE  21 


QUESTION  24:  TE1 ECONFERENCING  EQUIPMENT 
EASY  TO  OPERATE 

Class  Agree  %  Undecided  %  Disagree  % 


TEDS  CT 

(R) 

35.0 

25.0 

40.0 

TEDS  DT 

(R) 

60.0 

35.0 

5.0 

(SL  = 

.0288* 

MO  =  13%) 

TEDS  CT 

56.4 

16.4 

27.3 

TEDS  DT 

71.4 

16.1 

12.5 

(SL  = 

.1326 

MO  =  28%) 

Remotes 

CT 

68.6 

11.4 

20.0 

Remotes 

DT 

77.8 

5.6 

16.7 

(SL  = 

.  5953 

MO  =  35%) 

*  Significance  <  .05 


TABLE  22 

QUESTION  25:  I'D  TAKE  ANOTHER  COURSE 
USING  TELETEACH 


Class 

Agree  % 

Undecided 

Disagree  % 

TEDS  CT  (R) 

30.0 

25.0 

45.0 

TEDS  DT  (R) 

80.0 

15.0 

5.0 

(SL  = 

.0033* 

MO  = 

13%) 

TEDS  CT 

41.8 

18.2 

40.0 

TEDS  DT 

S  8 . 9 

23.2 

17.9 

(SL  = 

.  0356* 

MO  = 

28%) 

Remotes  CT 

48.6 

14.3 

37.1 

Remotes  DT 

47.2 

27.8 

25.0 

(SL  - 

.  3042 

MO  = 

35%) 
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between  the  resident  classes  of  each.  Apparently,  since  the 
remotes  were  not  significantly  different,  most  of  the  differ¬ 
ence  was  caused  by  the  strong  agreement  by  resident  TEDS  DT 
versus  the  more  negative  attitude  by  the  resident  TEDS  CT 
class  (see  later  discussion) . 

Responses  to  Question  26,  "There  should  be  more 
interaction  among  the  sites,"  are  displayed  in  Table  23. 
Analysis  of  Question  26  responses  indicated  that  the  resident 
class  of  TEDS  CT  differed  significantly  from  the  resident 
class  of  TEDS  DT.  Resident  TEDS  CT  favored  more  site  inter¬ 
action,  but  resident  TEDS  DT  was  undecided.  Generally,  very 
few  disagreed,  but  many  were  undecided. 


TABLE  23 

QUESTION  26:  THERE  SHOULD  BE  MORE 
SITE  INTERACTION 


Class 

Agree  % 

Undecided  % 

Disagree  % 

TEDS  CT  (R) 

50.0 

30.0 

20.0 

TEDS  DT  (R) 

35.0 

65.0 

0 

(SL  * 

.0286* 

MO  *  .13%) 

TEDS  CT 

40.0 

45.5 

14.5 

TEDS  DT 

35.7 

55.4 

8.9 

(SL  - 

.4913 

MO  «  28%) 

Remotes  CT 

34.3 

54.3 

11.4 

Remotes  DT 

36.1 

50.0 

13.9 

(SL  = 

.9212 

MO  *  351) 

*  Significance  < 

.05 
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Responses  to  Question  27,  "The  absence  of  eye  contact 
with  the  instructor  created  a  learning  barrier/problem,"  are 
displayed  in  Table  24. 


TABLE  24 

QUESTION  27:  ABSENCE  OF  EYE  CONTACT 
LEARNING  PROBLEM 


Class 

Agree  % 

Undecided  % 

Disagree  % 

Remotes 

CT 

74.3 

11.4 

14.3 

Remotes 

DT 

72.2 

5.6 

22.2 

(SL  =  .5104 

MO  =  35%) 

Analysis  of  Question  27  responses  which  were  relevant 
only  to  the  combined  remote  groups  of  each  TEDS  class  revealed 
no  significant  difference.  As  might  be  expected,  both  groups 
considered  that  the  absence  of  eye  contact  with  the  instructor 
created  a  learning  problem.  This  feeling  is  not  substantiated 
by  post-test  scores  since  no  difference  was  found  between  the 
various  combinations  when  mean  post-test  scores  or  achievement 
were  examined  (see  Test  Score  Analysis  Results) . 

Further  examination  of  student  written  comments  indi¬ 
cated  that  many  students  felt  that  communication  was  hampered 
because  the  information  normally  transmitted  between  students 
and  instructor  by  body  language  was  missing.  For  example, 
instructors  could  not  determine  when  remote  students  had 
"puzzled  looks"  on  their  faces.  Additionally,  persistent  audio 
problems  mentioned  in  students  written  comments  probably  con¬ 
tributed  to  this  attitude. 
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Responses  to  Questions  28,  "The  person  locally  aid¬ 
ing  the  course  director  (site  monitor)  appeared  knowledgeable 
of  equipment  operation":  29,  "The  local  person  aiding  the 
course  director  (site  monitor)  had  the  room  and  materials  pre¬ 
pared  for  class";  and  30,  "A  subject  matter  expert  should  be 
available  at  each  site  to  assist  students",  are  displayed  in 
Tables  25,  26,  and  27. 

Analysis  of  Questions  28,  29,  and  30  responses  indi¬ 
cated  no  significant  difference  between  the  remotes  of  TEDS 
CT  and  DT.  Remote  students  thought  the  site  monitor  was 
knowledgeable  and  the  monitor  had  the  room  prepared.  They 
also  thought  that  a  subject  matter  expert  should  be  available 
at  each  site. 

Examination  of  the  written  responses  on  the  back  of 
the  end-of-course  answer  sheets  indicated  that  several  stu¬ 
dents  in  the  nonTEDS  offering  perceived  that  the  reorganiza¬ 
tion  of  SYS  223  resulted  in  some  confusion  in  the  course 
structure.  But  more  profoundly,  the  students  in  the  TEDS  CT 
offering,  even  the  resident  TEDS  CT  group,  were  uncomfortable 
with  the  organization  and  the  new  delivery  medium.  They  also 
perceived  that  the  guest  speakers  were  uncomfortable  with  the 
new  delivery  mode.  Although  there  were  some  negative  comments, 
the  TEDS  DT  class  had  a  more  positive  opinion. 

Several  other  important  aspects  surfaced  from  the 
TEDS  CT  and  DT  student  comments  which  were  supported  by  the 
course  director.  First,  many  technical  problems  occurred 
during  both  the  TEDS  CT  and  DT  offerings.  Mainly,  these  were 
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TABLE  25 


QUESTION  28:  SITE  MONITOR  KNOWLEDGEABLE 
OF  EQUIPMENT 


Class 

Agree  % 

Undecided  % 

Disagree  % 

Remotes 

CT 

77.1 

5.7 

17.1 

Remotes 

DT 

66.7 

16.7 

16.7 

(SL  =  .3371 

MO  =  351) 

Class 


TABLE  26 


QUESTION  29:  SITE  MONITOR  HAD 
ROOM  PREPARED 


Agree  % 


- ! 

Undecided  %  Disagree 


Remotes 

CT 

54.3 

8.6 

37.1 

Remotes 

DT 

55.6 

8.3 

36.1 

(SL  *  .9942 

MO  =  35%) 

TABLE  27 

QUESTION  30:  A  SUBJECT  MATTER  EXPERT 
SHOULD  BE  AVAILABLE 


Class 

Agree  % 

Undecided  % 

Disagree  % 

Remotes 

CT 

58.8 

26.5 

14.7 

Remotes 

DT 

58.3 

25.0 

16.7 

(SL  »  .9713 

MO  *  35%) 
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garbled  audio  transmissions  and  mechanical  malfunctions.  In 
addition,  there  were  numerous  comments  that  the  electronic 
blackboard  was  used  very  little  for  actual  instruction  in  the 
course.  Second,  the  course  director  stated  that  he  spent 
additional  time  explaining  to  the  TEDS  DT  class  what  was  ex¬ 
pected  of  them  and  some  of  the  idiosyncracies  of  the  TEDS 
equipment  (8) .  This  may  have  accounted  for  the  100  percent 
of  the  expectations  of  TEDS  DT  being  met. 

Student  Acceptance  Analysis  Results.  The  mean 
acceptance  scores  for  combined  Questions  23  and  25  are  in 
Table  28.  Scores  below  four  indicate  acceptance,  while  scores 
above  four  indicate  nonacceptance.  A  score  of  four  indicates 
undecided.  When  reviewing  the  results  presented  in  this 
chapter,  the  reader  should  remember  that  the  higher  the  num¬ 
erical  acceptance  scores,  the  less  satisfied  students  were 
with  TEDS.  Therefore,  increasing  acceptance  scores  mean 
decreasing  satisfaction  with  TEDS. 

Analysis  of  the  TEDS  class  combinations  indicated  a 
significant  difference  with  two  of  the  combinations.  First, 
the  TEDS  CT  class  had  a  mean  significantly  different  from 
TEDS  DT  class.  TEDS  CT  appeared  to  have  a  slightly  negative 
attitude  toward  TEDS  (4.2),  while  TEDS  DT  had  a  positive 
attitude  (3.1).  The  previous  discussion  concerning  the  dif¬ 
ferences  between  TEDS  CT  and  TEDS  DT  may  account  for  some  of 
the  improvement  in  TEDS  DT's  attitude  towards  Teleteach. 
Second,  there  was  a  significant  difference  in  means  when  some 
remote  sites  were  compared  against  each  other.  The  ESD  remote 
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TABLE  28 


MEAN  TELETEACH  ACCEPTABILITY  SCORES 
Class  Mean  (Scale:  0  to  8) 


TEDS  CT 

(R) 

5.0 

TEDS  DT 

(R) 

2.5 

(SL 

=  .3654 

MO  =  13?) 

TEDS  CT 

4.2 

TEDS  DT 

3.1 

(SL 

=  .0068* 

MO  =  28.4%) 

Remotes 

CT 

3.8 

Remotes 

DT 

3.5 

(SL 

=  .6009 

MO  =  34.9%) 

Remotes  by  Location  CT/DT: 


SD 

2.9 

AD 

ESD 

4.8 

HQ 

3.0 

SD 

2.8 

AD 

3.5 

ESD 

2.9 

HQ 

6.8 

(SL  -  .0044* 

MO  -  34.9%) 

Population  Mean  3.7 
*  Significance  <  .05 
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class  of  TEDS  CT  and  the  HQ  class  of  TEDS  DT  had  negative 
opinions,  while  the  remaining  locations  had  positive  opinions. 
Available  data  were  insufficient  to  explain  these  results. 

The  most  dramatic  reversal  of  acceptance  was  that  of 
the  small  HQ's  classes  (six  students  for  the  TEDS  CT  and  five 
students  for  TEDS  DT) .  Their  switch  to  a  very  negative  atti¬ 
tude  for  the  TEDS  DT  class  is  contrary  to  the  overall  trend 
and  unexplainable  with  available  data.  Their  written  comments 
(DT)  were  also  negative  toward  Teleteach  equipment  and  the 
instructors.  The  resident  TEDS  CT  class  opinion  seems  to  be 
the  determinant  which  caused  the  negative  opinion  of  the  total 
TEDS  CT  class.  Apparently,  even  though  the  resident  TEDS  CT 
group  had  visual,  in-person  contact  with  the  instructor,  they 
felt  that  the  remote  students  were  at  a  disadvantage  (8) .  At 
any  rate,  it  is  important  to  recognize  that  the  majority  of 
the  TEDS  CT  students  who  were  the  remote  students  found  TEDS 
acceptable . 

The  multiple  regression  results  for  the  three  TEDS 
combinations  (5,  6,  and  7  which  exclude  nonTEDS)  to  which  the 
acceptance  variable  applies  are  displayed  in  Table  29.  Analy¬ 
sis  of  Table  29  indicated  that  no  non-location  variables  were 
statistically  significant  as  independent  variables  in  a  linear 
relationship  when  acceptance  was  the  dependent  variable,  i.e., 
no  non-location  variable  was  a  significant  predictor  (at  the 
five  percent  level)  of  acceptance.  The  only  significant 
independent  variable  was  the  location  SD ,  an  indicator  variable. 
This  was  identified  when  regression  was  performed  on  the 
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TABLE  29 


REGRESSION  ANALYSIS:  ACCEPTANCE 


Combination 

Significant 

Predictor 

C 

B 

SL 

R2 

7 

SD 

3.88 

-1.31 

.  045 

.  07 

NOTE:  The  following  symbols  are  used  in  Tables  29,  39 
and  40. 

C  =  The  constant  term  in  the  regression  equation 

B  *  The  partial  regression  coefficient  (slope)  in 
the  regression  equation 

SL  =  Significance  level 

2 

R  =  Multiple  Coefficient  of  Determination  (ratio 
of  explained  variation  to  the  total  variation 
of  the  dependent  variable) 


remotes  of  TEDS  CT  and  DT  (combination  7) . 

Test  Score  Analysis  Results.  The  mean  scores  for 
the  three  test  score  variables  (pre-test,  post-test,  and 
achievement)  are  in  Tables  30  through  38.  Analysis  of  these 
tables  indicated  that  the  only  significant  differences  at  the 
five  percent  level  occurred  in  the  pre-test  and  achievement 
variables  when  nonTEDS  was  compared  to  the  TEDS  CT  by  loca¬ 
tion.  Although  the  ANOVA  statistic  does  not  exactly  show 
where  the  difference  lies,  observation  of  Table  30  and  Table 
31  (nonTEDS  to  TEDS  CT)  showed  SD  had  the  lowest  pre-test 
score  and  the  highest  achievement  score.  SD ' s  post-test  score 
was  close  to  the  population  mean,  but  their  low  pre-test  score 
made  it  possible  for  them  to  obtain  the  highest  achievement 
(post-test  minus  pre-test)  score.  It  is  noteworthy  that  the 
combined  mean  achievement  scores  for  the  remote  sites  of 


TABLE  30 


MEAN  TEST  AND  ACHIEVEMENT  SCORES  FOR 
COMBINATION  1 


Location 

Pre-test 

Post- test 

Achievement 

Population 

37.4 

73.4 

36.3 

nonTEDS 

38.4 

73.5 

35.1 

TEDS  CT 

36.9 

73 . 3 

37 . 2 

WPAFB 

36.2 

72.7 

36.5 

SD 

26.5 

72.4 

49.6 

AD 

36.4 

71.8 

36 . 0 

ESD 

43.4 

76.6 

30.8 

HQ 

44.6 

69.2 

34.2 

(MO  *  2.2% 

MO  =  12.5% 

MO  =  14%) 

TABLE  31 

SIGNIFICANCE 

LEVELS  FOR  TESTS 

OF 

DIFFERENCES 

FOR  COMBINATION 

1 

Combination 

Pre-test 

Post  -test 

Achievement 

nonTEDS 
to  TEDS  CT 

.6012 

.9296 

.4690 

By  Location 
(nonTEDS  ♦  5  .0446* 

TEDS  CT  sites) 

.  5324 

.0371* 

*  Significance  <  .05 
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TABLE  32 


MEAN  TEST  AND  ACHIEVEMENT  SCORES 
FOR  COMBINATION  2 


Location 

Pre-test 

Post- test 

Achievement 

Population 

36.2 

72.6 

36 . 7 

nonTEDS 

38.4 

73.5 

35.1 

TEDS  DT 

34.5 

71.8 

38.2 

WPAFB 

31.0 

73.1 

42.4 

SD 

31.0 

74.0 

43.0 

AD 

38.1 

77.6 

41.1 

ESD 

36.5 

66.4 

30.8 

HQ 

35.0 

67.5 

32.6 

(MO  =3.5% 

MO  =  11.3% 

MO  =  13.9% 

TABLE  33 


SIGNIFICANCE  LEVELS  FOR  TESTS  OF 
DIFFERENCES  FOR  COMBINATION  2 


Combination 

Pre-test 

Post  -  test 

Achievement 

nonTEDS  to 

TEDS  DT 

.  2071 

.3810 

.3346 

By  Location 
(nonTEDS  +  5 

.  5865 

.0538 

.2615 

TEDS  DT  sites) 
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TABLE  34 


MEAN  TEST  AND  ACHIEVEMENT  SCORES  FOR 
COMBINATION  3 


Combination 

Pre-test 

Post-test 

Achievement 

Population 

37.7 

73.2 

35.5 

nonTEDS 

38.4 

73.5 

35.1 

TEDS  CT (R) 

36.2 

72.7 

36.5 

(SL 

=  .  5438 

SL  =  .7227 

SL  =  .6989) 

(MO 

II 

h-4 

-P* 

o'*4 

MO  =  0% 

MO  =  1.4%) 

TABLE  35 

MEAN 

TEST  AND  ACHIEVEMENT  SCORES  FOR 

COMBINATION  4 

Combination 

Pre-test 

Post- test 

Achievement 

Population 

36.2 

73.4 

37.1 

nonTEDS 

38.4 

73.5 

35.1 

TEDS  DT  (R) 

31.0 

73.1 

42.4 

(SL 

=  .0685 

SL  *  .8796 

SL  =  .0743) 

(MO 

=  4.3% 

MO  =  4.3% 

MO  =  7.1%) 

TEDS  CT  (37. 5)  and  TEDS  DT  (36.0)  were  very  close  to  nonTEDS 
(35.1)  and  to  the  resident  TEDS  CT  (36.5).  The  resident  class 
of  TEDS  DT  had  a  somewhat  higher  mean  achievement  score  (42.4). 
The  fact  that  there  were  no  statistically  significant  differ¬ 
ences  between  the  post-test  scores  for  TEDS  CT  and  any  of  the 
comparisons  was  an  interesting  finding  since  several  of  the 
student  end-of -course  responses  revealed  negative  (or  less 
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TABLE  36 


MEAN  TEST  AND  ACHIEVEMENT  SCORES  FOR 
COMBINATION  5 


Combination 

Pre-test 

Post-test 

Achievement 

Population 

33.8 

72.9 

39.0 

TEDS  CT  (R) 

36.2 

72.7 

36.5 

TEDS  DT  (R) 

31.0 

73.1 

42.4 

(SL  =  .1854 

SL  =  .8816 

SL  =  .2050) 

©\° 

ro 

II 

O 

MO  -  6.5% 

MO  =  8.7%) 

positive)  opinions  of  TEDS  by  the  TEDS  CT  group  in  the  areas 
of  course  structure,  handouts,  time  better  utilized  elsewhere 
course  met  expectations,  class  hours,  TEDS  acceptance,  and 
TEDS  equipment.  Also,  because  there  were  no  significant  diff 
erences  in  overall  achievement  scores  among  nonTEDS ,  TEDS  CT , 
and  TEDS  DT,  one  might  assume  that  the  negative  or  less  posi¬ 
tive  attitudes  by  the  TEDS  CT  class  were  not  reflected  in 
greater  achievement  results. 

To  summarize  the  test  score  analysis,  the  seven 
combinations  listed  in  Table  1  were  tested  by  one-way  ANOVA. 
No  significant  differences  were  found  in  pre-test  scores, 
post-test  scores,  and  achievement  scores  for  these  combina¬ 
tions.  However,  further  analysis  of  the  seven  combinations 
by  location  did  reveal  differences  among  the  locations  in¬ 
volved  in  the  comparison  of  combination  1  (nonTEDS  to 
TEDS  CT) . 
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TABLE  37 


MEAN  TEST  AND  ACHIEVEMENT  SCORES 
FOR  COMBINATION  6 


Combination 

Pre-test 

Post-test 

Achievement 

Population 

35. 9 

1  L.l 

37.6 

TED  CT 

36.9 

73.3 

37.2 

TEDS  DT 

34.5 

71.8 

38.2 

(SL  =  .3998 

SL  =  .3675 

SL  =  .7344) 

(MO  =  3.21 

MO  *  19.4% 

MO  =  21.3%) 

By  Location 

,  Ten  Sites 

TEDS  CT: 

WPAFB 

36.2 

72.7 

36.5 

SD 

26.5 

72.4 

49.6 

AD 

36.4 

71.8 

36.0 

ESD 

43.4 

76.7 

30.8 

HQ 

44.6 

69.2 

34.2 

TEDS  DT: 

WPAFB 

31.0 

73.1 

42.4 

SD 

31.0 

74.0 

43.0 

AD 

38.0 

77.6 

41.1 

ESD 

36.5 

66.4 

30 . 8 

HQ 

35.0 

67.5 

32.6 

(SL  *  .1378 

SL  =■  .0914 

SL  -  .0758) 

(MO  -  3,21 

MO  -  19.4% 

MO  *  21.3%) 

TABLE  38 


MEAN  TEST  AND  ACHIEVEMENT  SCORES 
FOR  COMBINATION  7 


Combination 

Pre-test 

Post-test 

Achievement 

Population 

36.7 

72.6 

36.8 

TEDS  CT 
(Remotes) 

37.1 

73.6 

37.5 

TEDS  DT 
(Remotes 

36.1 

71.1 

36.0 

(SL  =  .7763 

SL  =  .2273 

SL  =  .6939) 

(MO  =  2.8% 

o\° 

OO 

Tt 

rq 

O 

s 

MO  =  26.6%) 

The  stepwise  multiple  regression  results  are  displayed 
in  Tables  39  and  40.  (The  key  beneath  Table  29  explains  the 
symbols  used  for  these  tables.)  Table  39  shows  the  results 
when  regression  was  performed  using  post-test  scores  as  the 
dependent  variable.  Table  40  shows  the  results  when  regres¬ 
sion  was  performed  using  achievement  as  the  dependent  variable. 

The  regression  analysis  of  the  post-test  variable 
yielded  several  predictors  at  the  five  percent  significance 
level.  First,  regression  for  the  combined  nonTEDS  and  TEDS 
CT  classes  (Table  39,  combination  1)  indicated  that  age  was 
the  only  significant  independent  variable  in  a  linear  rela¬ 
tionship  with  post-tests  as  the  dependent  variable,  i.e.,  a 
significant  predictor  of  post-tests.  A  close  examination  of 
Table  39  (combination  1)  reveals  that  as  the  age  category  (or 
factor)  increased,  post-test  scores  decreased,  The  "R" 
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TABLE  39 


REGRESSION  ANALYSIS:  POST-TEST  SCORES 


Combination 

Significant 

Predictor 

C 

B 

SL 

R2 

1 

Age 

76.1 

-4.9 

.002 

.21 

2 

Experience 

71.8 

+  2.6 

.009 

.13 

ESD 

-7.7 

.012 

.12 

5 

Age 

76.0 

-5.3 

.006 

.10 

Grade/Rank 

+  3.4 

.038 

.10 

6 

Experience 

75.2 

+  2.2 

.001 

.07 

Age 

-3.2 

.002 

.10 

7 

Experience 

74.3 

+  3.2 

.001 

.16 

Age 

-3.4 

.007 

.11 

NOTE:  See 

Table  29  (page 

65)  for 

definition  of  symbols. 

square  score  indicated  that  the  age  factor  could  explain  about 
21  percent  of  the  variation  from  the  mean  score.  For  combina¬ 
tion  2  (nonTEDS  and  TEDS  DT) ,  regression  revealed  experience 
as  the  only  significant  predictor  of  post-test  performance. 

The  indicator  (or  location)  variable  ESD  was  also  significant. 
As  the  experience  category  increased,  post-test  scores  in¬ 
creased.  Combined,  these  predictors  explain  about  25  percent 
of  the  variance  about  the  mean. 

For  combinations  3  and  4,  there  were  no  significant 
predictors  for  post-test  performance.  Regression  for  com¬ 
bination  5  (resident  TEDS  CT  and  resident  TEDS  DT)  yielded 
age  and  grade/rank  as  significant  predictors  for  post-test 
performance.  As  the  age  category  increased,  post-test  scores 
decreased.  As  the  grade/rank  category  increased,  post-test 


TABLE  40 


REGRESSION  ANALYSIS:  ACHIEVEMENT  SCORES 


Combination 

Significant 

Predictor 

C 

B 

SL 

1 

Experience 

41.4 

-4.6 

.004 

.20 

2 

Age 

46.0 

-5.4 

.020 

.11 

3 

Age 

52.3 

-9.0 

.049 

.20 

5 

Age 

49.9 

-7.1 

.007 

.19 

6 

Experience 

54.6 

-3.0 

.011 

.  20 

Age 

54.6 

-4.2 

.016 

.03 

7 

Experience 

52.7 

-4.3 

.002 

.17 

Acceptance 

52.7 

-2.1 

.035 

.07 

NOTE:  See  Table  29  (page 

65)  for 

definition  of 

symbols . 

scores  increased.  Age  and  grade/rank  explain  only  20  percent 
of  the  post-test  variance. 

For  both  the  last  two  combinations,  combined  TEDS  CT 
and  TEDS  DT  and  the  combined  remotes  of  both  TEDS  classes, 
regression  yielded  experience  and  age  as  statistically  signi¬ 
ficant  predictors.  As  the  experience  category  increased, 
post-test  scores  increased.  As  the  age  category  increased, 
post-test  scores  decreased. 

Earlier  demographic  analysis  revealed  no  significant 
differences  with  any  combinations  for  age  or  experience. 
Apparently,  the  strongest  trend  was  for  the  younger  students 
to  score  higher  on  the  post-tests.  Also,  experience  appeared 
in  three  of  the  regressions  as  a  significant  predictor.  As 
might  be  expected,  the  more  experienced  students  tended  to 
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have  the  higher  post-test  scores. 


Regression  analysis  for  the  achievement  variable 
(Table  40)  revealed  three  significant  predictors  at  the  five 
percent  significance  level.  Age  appeared  as  a  significant 
predictor  in  combinations  2,  3,  5,  and  6.  In  each  case,  as 
the  age  category  increased,  the  achievement  score  decreased. 
This  parallels  the  post-test  results  previously  discussed. 

The  younger  students  tended  to  score  higher  on  the  post-test 
and  achievement  scores. 

On  the  other  hand,  experience,  which  was  a  signifi¬ 
cant  predictor  for  combinations  1,  6,  and  7,  had  the  opposite 
effect  on  achievement  scores  as  it  had  on  post-test  scores. 

As  the  experience  category  increased,  achievement  scores 
decreased . 

The  third  significant  predictor,  acceptance,  was 
identified  when  the  remotes  of  the  two  TEDS  classes  were  com¬ 
bined.  Earlier  analysis  of  acceptance  scores  showed  that  the 
majority  of  both  classes  slightly  favored  TEDS.  As  stated 
earlier,  the  higher  the  acceptance  score  (on  the  scale  of 
0  to  8) ,  the  less  satisfaction  with  TEDS.  As  acceptance 
scores  increased  (less  satisfaction  with  TEDS) ,  achievement 
scores  decreased.  This  finding  indicated  that  those  TEDS 
students  less  satisfied  with  TEDS  had  lower  achievement  scores 
than  those  satisfied  with  TEDS. 

Earlier  analysis  revealed  no  significant  difference 
within  the  appropriate  combinations  for  age,  experience,  or 
acceptance.  Of  course,  the  acceptance  variable  was  only 
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applicable  to  combinations  which  excluded  nonTEDS  (combina¬ 
tions  5 ,  6 ,  and  7)  . 


To  summarize  the  regression  analysis  results  for 
achievement  scores,  the  following  variables  were  found  to  be 
important  predictors  in  one  or  more  of  the  seven  combinations: 
age,  experience,  and  acceptance.  In  general,  the  younger, 
less  experienced  students  were  predicted  to  score  somewhat 
higher  on  the  achievement  score.  This  appears  logical  since 
the  measure  of  achievement  used  was  the  difference  between 
pre-tests  and  post-tests. 

Instructor  Acceptance  Analysis  Results.  Table  41 
displays  the  responses  to  the  instructor  questionnaire.  The 
questions  are  listed  in  Appendix  C.  The  total  number  of 
instructors  responding  to  a  question  was  31  unless  stated 
otherwise  in  the  table.  When  helpful,  comments  that  instruc¬ 
tors  wrote  on  the  back  of  the  answer  sheet  are  summarized  to 
support  the  analysis.  As  described  in  Chapter  II,  greater 
than  SO  percent  response  is  used  to  determine  if  instructors 
had  a  definite  opinion. 

Analysis  of  Table  41  indicated  that  instructors  had 
a  definite  opinion  on  Questions  1,  4,  6,  and  11.  Instructors 
felt  that  equipment  problems,  mainly  audio  transmissions,  and 
lack  of  face-to-face  contact  were  a  problem  during  course 
presentations  when  using  TEDS.  Their  written  comments  indi¬ 
cated  that  TEDS  required  different  presentation  techniques 
because  of  that  lack  of  face-to-face  contact,  that  their 
usual  "free  flowing"  presentation  was  inadequate,  and  that 
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TABLE  41 


INSTRUCTOR  CRITIQUE  RESPONSES 


Questions  (N*) 

Agree  % 

Undecided  % 

Disagree  % 

1 

32.2 

22.5 

45 

2 

61.2 

3.2 

35.4 

3 

41.9 

9.6 

48.3 

4 

19.3 

19.3 

61.2 

S 

35.4 

32.3 

32.2 

6 

54.8 

16.1 

29 

7 

51.6 

12.9 

35.4 

8 

29 

25.8 

45 

9 

(30) 

23.3 

33.3 

43.3 

10 

(30) 

43.3 

20 

36.6 

11 

(30) 

23.3 

60 

16.6 

121 

**  (27) 

7.4 

59.2 

33.3 

15 

(26) 

23 

50 

26.9 

16 

(25) 

48 

20 

32 

17 

(25) 

24 

52 

24 

*  N  is  the  number  of  instructors  responding  to  a  question; 

N  *  31  unless  another  number  is  given  in  (  ) . 

**Questions  13  and  14  were  omitted. 

lack  of  face-to-face  contact  made  student  rapport  difficult 
to  establish.  Also,  a  few  stated  that  TEDS  degrades  the 
quality  of  instruction  at  AFIT  because: 

1.  Lesson  objectives  must  be  greatly  reduced. 

2.  Student  boredom  is  high  due  to  the  slow  pace. 
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3.  Absenteeism  is  high  at  remote  sites 

4.  Instructor  enthusiasm/motivation  is  reduced. 

Instructors  disagreed  (Question  4)  that  student  par¬ 
ticipation  from  remote  sites  met  or  exceeded  their  expecta¬ 
tions.  Even  though  some  written  comments  suggested  otherwise, 

60  percent  responded  neutral  to  Question  11  concerning  whether 
TEDS  caused  a  change  in  instructor  presentation  technique. 

The  fact  that  many  instructors  added  that  it  had  been  too 
much  time  since  they  gave  a  presentation  for  them  to  remember 
their  feelings  probably  contributed  to  neutral  response  here 
and  elsewhere.  No  definite  opinion,  according  to  the  SO  per¬ 
cent  rule,  was  expressed  on  the  remaining  questions. 

Even  though  it  must  be  emphasized  that  written  comments 
were  expressed  by  a  minority,  they  seem  to  offer  additional 
insight  into  the  analysis.  Two  other  comments  were  relevant. 
First,  the  suggestion  was  made  several  times  that  implementa¬ 
tion  of  closed-circuit  TV  would  alleviate  the  problems  caused 
by  lack  of  face-to-face  contact.  Second,  some  instructors 
stated  that  TEDS  could  be  very  effective  when  instructors  be¬ 
come  more  familiar/comfortable  with  the  system. 

The  acceptance  of  TEDS  by  instructors  was  directly 
determined  by  Question  8.  Even  though  more  instructors  felt 
that  TEDS  was  unacceptable  (as  judged  by  their  not  wanting 
to  teach  again  using  TEDS) ,  no  clear  opinion  was  expressed  in 
the  three  responses. 


CHAPTER  IV 


CONCLUSIONS  AND  RECOMMENDATIONS 

In  this  chapter  the  results  of  Chapter  III  are  eval¬ 
uated  in  light  of  the  research  objectives,  hypotheses,  and 
decision  criteria  previously  described  in  Chapter  II.  Con¬ 
clusions  drawn  from  the  research  data  are  presented  first. 

Then  recommendations  aimed  at  course  improvements  and  future 
research  are  presented.  Because  this  research  effort  analyzed 
only  three  1980  offerings  of  SYS  223,  these  findings  do  not 
necessarily  apply  across  the  SYS  223  program  cr  to  other 
course  offerings  using  the  Teleteach  Expanded  Delivery  System. 

Conclusions 

Demographic  Results.  The  decision  rule  used  to 
determine  whether  the  student  groups  were  comparable  in  terms 
of  the  demographic  variables  was:  If  the  contingency  table 
analysis  yields  no  statistical  differences  among  the  compari¬ 
son  groups,  accept  the  null  hypothesis  that  the  groups  were 
homogeneous . 

The  contingency  table  analysis  revealed  one  statis¬ 
tically  significant  difference  with  demographic  variables, 
which  was  the  grade/rank  distribution  between  the  non TEDS  and 
TEDS  CT  groups.  Therefore,  the  null  hypothesis  for  grade/rank 
was  rejected.  However,  considering  the  results  of  acceptance 
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and  test  score  analysis,  this  difference  was  not  considered 
to  be  of  significance  in  the  overall  evaluation,  i.e.  grade/ 
rank  was  not  found  to  be  a  predictor  for  acceptance  or  any 
test  score  variables  within  the  nonTEDS  to  TEDS  CT  comparison. 
Since  no  significant  differences  were  found  on  other  demo¬ 
graphic  variables,  the  decision  was  made  to  accept  the 
remaining  null  hypotheses  for  demographic  variables. 

Student  Acceptance  of  TEDS.  The  decision  rule  used 
to  determine  whether  TEDS  was  acceptable  to  the  TEDS  students 
was:  If  the  mean  value  of  the  responses  to  the  end-of -course 

questions  selected  to  determine  acceptability  indicate  that 
a  majority  of  students  found  TEDS  acceptable,  accept  the  null 
hypotheses  that  the  TEDS,  as  used  in  SYS  223,  was  acceptable 
to  the  students. 

The  results  of  the  mean  acceptability  score  permit 
rejection  of  the  null  hypotheses  for  TEDS  CT  and  resident 
TEDS  CT  groups.  In  short,  these  two  groups  did  not  accept 
TEDS.  However,  for  the  remotes  of  TEDS  CT  and  all  the  TEDS 
DT  comparisons  (resident  and  remote) ,  the  null  hypotheses  are 
accepted,  i.e.,  TEDS  is  acceptable. 

Overall,  the  majority  of  the  students  in  both  TEDS 
CT  and  TEDS  DT  considered  the  delivery  medium  acceptable. 

This  is  not  to  say,  however,  that  there  was  no  dissatisfaction 
with  the  system  or  the  course.  Considerable  dissatisfaction 
was  expressed  in  responses  to  some  of  the  end-of -course 
questions  and  through  written  comments. 


Based  upon  the  responses  to  the  end-of -course 
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questions,  the  different  combinations  of  TEDS  CT  found  fault 
with  the  course  structure,  their  ability  to  ask  questions, 
the  quantity  of  handouts,  the  wise  use  of  their  time,  the 
course  hours,  and  equipment  operation.  On  the  other  hand, 
the  data  analysis  of  TEDS  DT  leads  the  researcher  to  conclude 
that  many,  of  these  shortcomings  were  overcome  for  the  TEDS  DT 
offering.  This  conclusion  was  based  upon  the  more  positive 
responses  by  TEDS  DT  students  on  the  structured  portion  and 
the  open-ended  portion  (student  comments)  of  the  end-of- 
course  questionnaire.  Further  support  for  this  position  was 
forthcoming  during  an  interview  with  the  course  director. 

More  than  70  percent  in  both  TEDS  remote  classes  felt 
that  the  absence  of  eye  contact  with  instructors  created  a 
learning  barrier.  This  feeling  was  not  supported  by  the  test 
results.  The  mean  post-test  scores  and  mean  achievement  scores 
were  very  close  between  all  groups  who  had  eye  contact  with 
the  instructor  (nonTEDS,  resident  TEDS  CT,  and  resident  TEDS 
DT)  and  those  groups  who  had  no  eye  contact  (remote  TEDS  CT 
and  DT) . 

Student  Acceptance  of  the  TEDS  Schedule.  The  deci¬ 
sion  rule  used  to  determine  whether  the  TEDS  schedule  was 
acceptable  to  students  was:  If  group  responses  were  greater 
than  50  percent  in  agreement  (disagreement)  with  Question  17, 
conclude  that  the  TEDS  schedule  was  acceptable  (unacceptable) . 
As  evidenced  by  the  responses  to  Question  17,  resident  TEDS 
CT ,  TEDS  CT,  and  the  remotes  of  TEDS  DT  did  not  like  the 
schedule.  The  remaining  TEDS  groups  did  not  express  a  clear 
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opinion,  but  they  had  slightly  less  than  50  percent  that  dis¬ 
liked  the  hours.  The  nonTEDS  class  was  clearly  pleased  with 
their  hours.  Despite  the  time  zone  differences,  none  of  the 
TEDS  classes  found  the  TEDS  schedule  acceptable.  Analysis  of 
written  comments  suggested  that  TEDS  classes  disliked  the 
hours  because  their  supervisors  expected  too  much  of  them  on 
their  normal  jobs  in  addition  to  attending  class.  In  light 
of  the  above,  the  researcher  felt  that  it  was  probably  not 
the  hours  the  course  was  offered,  but  the  fact  that  the  stu¬ 
dents  were  required  to  work  at  their  jobs  when  not  in  class 
which  probably  caused  the  negative  opinions  expressed  by  TEDS 
students  concerning  the  schedule. 

Learning  Effectiveness  of  TEDS.  The  decision  rule 
used  to  determine  whether  the  research  objective  of  learning 
effectiveness  was  met  was:  If  there  were  no  statistically 
significant  differences  with  the  mean  achievement  scores  of 
the  nonTEDS  class  and  the  TEDS  classes,  accept  the  null  hypo¬ 
thesis  that  learning  effectiveness  by  the  TEDS  method  is  no 
different  from  learning  effectiveness  of  the  nonTEDS  method 
of  instruction.  Except  when  compared  by  location,  the  analy¬ 
sis  of  the  test  results  in  Chapter  III  revealed  no  statisti¬ 
cally  significant  differences  in  the  learning  effectiveness 
as  measured  by  post- test  scores  or  achievement  scores.  In 
addition,  no  significant  differences  were  found  with  any  of 
the  seven  combinations  (as  groups).  Therefore,  evidence 
supported  the  hypothesis  that  the  TEDS  method  of  instruction 
was  as  effective  as  the  nonTEDS  method  of  instruction.  This 
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finding  is  quite  interesting  when  one  considers  the  equipment 
malfunctions  and  the  negative  attitudes  which  were  identified 
earlier. 

The  only  significant  differences  occurred  in  the 
pre-test  and  achievement  scores  when  the  nonTEDS  and  TEDS  CT 
comparisons  were  further  analyzed  by  location.  However,  these 
differences  were  not  large  enough  to  cause  significant  differ¬ 
ences  in  the  nonTEDS  to  TEDS  CT  comparison.  Thus,  no  evidence 
was  found  that  suggests  the  learning  effectiveness  of  TEDS  was 
different  among  the  nonTEDS,  TEDS  CT,  and  TEDS  DT  classes. 

Regression  analysis  indicated  that  age,  experience, 
and  acceptance  were  significant  predictors  of  achievement. 

Each  comparison  group  had  approximately  the  same  percentage 
of  students  in  each  category  and  did  not  differ  significantly 
on  any  of  these  factors.  Even  if  the  achievement  "measure" 
were  to  include  one  or  more  of  these  predictors,  there  would 
probably  be  no  significant  difference  in  average  achievement 
because  the  comparison  groups  did  not  differ  significantly 
on  any  of  these  three  factors. 

In  conclusion,  post-test  and  achievement  results  sup¬ 
port  the  contention  that  students  learn  as  well  when  instructed 
by  TEDS  or  nonTEDS. 

Additionally,  there  were  no  significant  differences 
in  aggregate  test  performance  results  between  the  resident 
TEDS  students  who  had  the  advantage  of  face-to-face  contact 
with  the  instructor  and  the  remote  TEDS  students  whose  contact 
with  the  instructor  was  limited  to  audio  and  blackboard  video. 
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Among  the  remote  locations,  the  differences  in  achievement 
were  attributable  primarily  to  differences  in  pre-test 
scores.  Although  greater  than  50  percent  of  the  students  and 
instructors  perceived  the  lack  of  eye  contact  to  be  a  learning 
barrier,  measured  learning  was  not  impaired. 

Instructor  Acceptance  of  TEDS.  The  decision  rule 
used  to  determine  whether  TEDS,  as  used  in  SYS  223,  was 
acceptable  to  instructors  was:  If  greater  than  50  percent 
of  instructor  responses  are  in  agreement  (disagreement)  with 
Question  8  on  the  instructor  questionnaire,  accept  the  null 
hypothesis  that  instructors  consider  TEDS  an  acceptable 
(unacceptable)  delivery  system.  The  analysis  of  instructor 
end-of -course  Question  8  responses  indicated  that  45  percent 
of  the  participating  instructors  found  TEDS  not  to  their 
liking.  According  to  the  decision  rule  used,  instructors 
expressed  no  clear  opinion  on  TEDS  acceptance.  Only  29  per¬ 
cent  of  the  instructors  indicated  that  they  would  like  to 
use  TEDS  again.  From  their  responses  to  the  questionnaire 
and  from  written  comments  on  the  back  of  the  answer  sheet, 
it  was  clear  that  many  instructors  felt  that  TEDS  impaired 
learning.  However,  as  previously  discussed,  test  results  do 
not  support  this  opinion. 

While  not  all  questions  concerning  the  use  of  TEDS 
have  been  answered,  the  central  research  objectives  have  been  met. 
Based  on  this  analysis,  TEDS  did  not  adversely  effect  learning; 


the  majority  of  TEDS  students  considered  it  acceptable;  the 
instructors  presented  no  clear  consensus  on  TEDS  acceptance. 
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Recommendations 


Even  though  the  conclusions  reached  were  limited  to 
three  specific  offerings  of  SYS  223,  the  researcher  can  make 
some  recommendations  which  might  be  helpful  to  the  overall 
TEDS  program  and  to  future  TEDS  research. 

This  study  revealed  several  areas  where  the  current 
TEDS  system  could  be  improved.  First,  students  should  be 
given  an  initial  briefing  on  the  positive  and  negative  aspects 
of  TEDS.  This  "up  front"  approach  to  explaining  TEDS,  as  was 
done  in  TEDS  DT,  will  hopefully  dispel  false  assumptions 
(such  as  loss  of  eye  contact  impeding  learning)  and  allow 
the  students  to  concentrate  on  the  message  of  the  course 
rather  than  its  medium.  During  such  a  briefing,  each  stu¬ 
dent’s  responsibilities  should  be  thoroughly  explained. 

The  instructors  should  be  better  prepared  to  teach 
the  course  using  TEDS.  It  is  essential  for  the  course  organi¬ 
zation  and  structure  to  be  well  sychronized  when  using  TEDS. 
Particularly  when  guest  instructors  are  used,  they  should 
understand  the  positive  aspect  and  the  peculiarities  asso¬ 
ciated  with  TEDS.  For  example,  they  should  understand  that 
TEDS'  ability  to  reach  more  students  and  to  save  money  and 
time  (4:105)  should  outweigh  the  disadvantages  of  having  to 
proceed  at  a  slower  pace  and  modify  their  presentation  styles. 
A  training  program  could  be  established  for  this  purpose. 
Perhaps  such  a  program  would  improve  instructor  acceptance. 

Third,  because  the  number  of  students  at  each  re¬ 
mote  site  is  relatively  small,  it  is  important  for  course 


directors  to  insure  that  the  data  collection  process  is 
complete,  timely,  and  accurate  in  order  for  statistical 
analysis  to  be  effective. 

Fourth,  it  is  also  critical  that  course  directors 
and  site  monitors  be  aware  of  the  importance  of  distribution 
of  instructional  aids  such  as  vugraphs  and  course  materials. 
If  materials  are  not  available  when  the  classes  are  held, 
the  students  may  have  difficulty  following  the  instruction. 

Fifth,  supervisors  should  be  advised  to  treat  the 
students  as  if  they  were  TDY  during  class  hours.  The  stu¬ 
dents  should  not  be  expected  to  perform  their  normal  duties. 

Sixth,  classes  at  each  site  should  have  as  many 
students  enrolled  as  each  classroom  can  accommodate  properly. 

Seventh,  the  regression  results  of  this  thesis  (see 
Chapter  III)  indicates  that  some  demographic  factors  were 
significant  predictors  of  post-test  scores.  An  alternative 
definition  of  achievement  could  contain  consideration  of 
these  significant  factors.  In  this  way,  TEDS  achievement 
comparisons  might  better  isolate  the  effect  of  TEDS  alone  on 
academic  achievement. 

Eighth,  future  studies  of  TEDS  might  address  student 
performance  on  less  objectively  measured  activities,  including 
team  and  individual  reports,  exercises,  simulations,  and  case 
studies . 

Finally,  data  collection  instruments  might  be 
developed  which  address  specific  concerns  in  order  to  deter¬ 
mine  definitive  cause-effect  relationships  in  areas  such  as 
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schedule/supervisor  expectations,  classroom  environment/ 
performance  and  acceptance,  and  course  completion  effect  upon 
specific  job  performance. 
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appendix  a 

DEMOGRAPHIC  INFORMATION 


ANSWER  EITHER  3  or  4 
NOT  BOTH 


DEMOGRAPHIC  INFORMATION 


GENERAL  INSTRUCTIONS  FOR  COMPLETING  THE  SURVEY 


Use  the  attached  answer  sheet  to  mark  your  responses.  Use  only  a  No.  2 
pencil  when  filling  out  the  answer  sheet.  DO  NOT  USE  INK.  Enter  your 
4-digit  student  number  in  the  last  four  positions  in  the  STUDENT  NUMBER 
area.  P lease  do  NOT^ write  your  name  or  social  security  number  anywhere 
on  the  answer  sheet.  Select  only  one  answer  co  each  question.  Mark  the 
answer  sheet  carefully  to  negate  computer  error.  Fill  in  the  box  with 
a  heavy  mark,  do  not  go  outside  the  lines  of  the  box.  If  you  made  a 
mistake,  erase  the  mark  completely  before  entering  a  new  one. 

1.  My  present  STATUS  is: 


A.  Officer 

B.  Enlisted 

C.  Civilian 

D.  Contractor 

E.  Other  (foreign,  etc.) 

2.  My  present  RANK  or  GRADE  is:  (If  you  answered  D  or  E  above,  please 
leave  blank.) 


Officer 

Enl is  ted 

Civilian 

A. 

01  or  02 

E4 

GS  5-10 

B. 

03 

E5 

GS  11 

C. 

04 

E6 

GS  12 

D. 

05 

E7 

GS  13 

E. 

06 

Eb-9 

GS  14 

My 

EDUCATIONAL  background: 

IMark  highest  completed) 

(Answer  only  question  3  0£ 

4,  not  both) 

A. 

Did  not 

complete  high  school 

B. 

High  school  graduate  or 

equivalent 

C. 

College- 

-some  credits 

D. 

Col  lege 

-  Associate  degree  (A. A.  or  A. Sc.) 

E. 

Co  1 1 ege 

-  Baccalaureate  degree  (B.A.  or  B.S.) 

4.  Continued  from  above 

A.  College  -  Graduate  credit,  no  graduate  degree 

B.  College  -  Master's  Degree 

C.  College  *  Work  beyond  Master's 

D.  College  -  Doctorate 

V  E.  None  of  the  above 

\ 
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5'  My  SEX  is: 


A.  Female 

B.  Male 

6.  My  present  AGE  is: 

A.  20-25 

B.  26-35 

C.  36-45 

D.  46-55 

E.  56  or  over 

7*  Years  of  EXPERIENCE  in  a  job  related  to  the  course: 

A.  0-1 

B.  2-3 

C.  4-5 

D.  6-7 

E.  8  or  more 


Answer  only  THREE  of  the 

next 

six 

questions,  8 

or  9,  10  or  11, 

12 

or 

The  FIRST  digit 

of  your 

OAFSC 

or 

Civilian 

Job 

Series  Number: 

Select  8.  A. 

0 

B. 

1 

C. 

2 

D.  3 

E. 

4 

One  9-  A. 

5 

B. 

6 

C. 

7 

D.  8 

E. 

9 

The  SECOND  digit 

:  of  your 

DAFSC  or 

Civilian 

i  Job 

Series  Number: 

Select  10.  A. 

0 

B. 

1 

C. 

2 

D.  3 

E. 

4 

One  11.  A. 

5 

B. 

6 

C. 

7 

D.  8 

E. 

9 

The  THIRD  digit 

of  your 

DAFSC 

or 

Civi 1 ian 

Job 

Series  Number: 

Select  12.  A. 

0 

B. 

1 

C. 

2 

0.  3 

E. 

4 

One  13-  A. 

5 

B. 

6 

C. 

7 

0.  8 

E. 

9 
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APPENDIX  B 

STUDENT  END-OF-COURSE  CRITIQUE 


STUDENT  END-OF-COURSE  CRITIQUE 

* 

This  critique  is  designed  to  obtain  feedback  concerning  whether  the 
course  achieved  its  objectives.  Your  daily  critiques  have  addressed 
most  of  the  specific  aspects  of  the  course.  Your  contribution  to  the 
improvement  of  this  course  is  greatly  appreciated  and  will  benefit 
future  students. 

Please  answer  each  question  to  the  best  of  your  ability.  Your  answer 
sheet  will  be  machine  processed  except  for  the  questions  in  Part  II. 
Additional  written  comments  are  welcomed. 

GENERAL  INSTRUCTIONS  FOR  COMPLETING  THE  SURVEY 

Use  the  attached  answer  sheet  to  mark  your  responses.  Use  only  a  No.  2 
pencil  when  filling  out  the  answer  sheet.  DO  NOT  USE  INK.  Enter  your 
four-digit  student  number  in  the  last  four  positions  of  the  STUDENT 
NUMBER  area.  Please  do  NOT  write  your  name  or  social  security  number 
anywhere  on  the  answer  sheet.  Select  only  one  answer  to  each  question. 
Mark  the  answer  sheet  carefully  to  negate  computer  error.  Fill  in  the 
box  with  a  heavy  mark;  do  not  go  outside  the  lines  of  the  box.  If  you 
make  a  mistake,  erase  the  mark  completely  before  entering  a  new  one. 

The  Part  II  questions  require  a  written  response.  Put  your  answers  on 
the  back  of  the  answer  sheet. 

PART  I 

Respond  by  using  the  options  A  thru  E  indicating  the  degree  to  which  you 
agree  with  the  statements  below. 

A.  Strongly  agree 

B.  Agree 

C.  Neither  agree  nor  disagree 

D.  Disagree 

E.  Strongly  disagree 

(These  options  will  be  repeated  at  the  top  of  each  page  for  your  conveni¬ 
ence.  ) 

1.  The  course  objectives  were  made  clear  either  orally  or  in  the  instruc 
tional  aids. 

2.  The  course  appeared  well  structured. 

3.  The  course  structure  permitted  questions  to  be  asked  and  answered 
sat isfactori ly. 

A.  The  room  was  conducive  to  learning. 

5.  I  was  in  a  position  where  I  could  hear  and  see  well. 
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A.  Strongly  agree 

B.  Agree 

#  C.  Neither  agree  nor  disagree 

D.  Disagree 

E.  Strongly  disagree 

6.  There  should  have  been  more  handout  materials. 

7.  The  course  should  have  been  longer. 

b.  My  time  could  have  been  better  utilized  elsewhere. 

9.  I  will  be  able  to  do  my  job  better  as  a  result  of  this  course. 

10.  The  course  met  my  expectations. 

11.  The  course  was  more  informative  than  I  had  anticipated. 

12.  Overall,  the  course  was  extremely  difficult. 

13-  Throughout  the  course,  there  was  adequate  transition  between  the 
various  days  of  instruction  In  terms  of  tying  in  and  relating  materials. 

14.  The  simulation  exercise  aided  in  the  total  learning  experience. 

(Darken  "c"  if  not  applicable.) 

15.  Discussion  of  the  tests  helped  me  learn. 

(Darken  "c“  if  not  applicable.) 

16.  The  tests  were  given  at  proper  intervals. 

17.  I  liked  the  hours  the  course  was  offered. 

a 

18.  I  learn  more  from  a  course  when  I  am  TDY  (completely  removed  from  my 
job  location). 

19.  When  required  to  critique  presentations,  I  learn  less  of  the  content 
presented. 

20.  The  "class  day"  should  be: 

A.  1-2  hours 

B.  2-3  hours 

C.  3-4  hours 
0.  4-5  hours 
E.  5-6  hours 

21.  How  many  presentations  were  you  unable  to  attend? 

A.  1-3  B.  4-6  C.  7-10  D.  11  or  more  E.  Had  perfect  attendance. 


92 


r 


22.  How  many  presentations  did  you  “make  up"  through  playback  of  the  audio* 
tapes? 

A.  1*3  B.  4-6  C.  7*10  D.  II  or  more  E.  None 

Use  these  responses: 

A.  Strongly  agree 
8.  Agree 

C.  Neither  agree  nor  disagree 
0.  Oisagree 
E.  Strongly  disagree 

23-  The  Teleteach  delivery  system  is  an  acceptable  learning  medium. 

24.  The  teleconferencing  equipment  (mikes  and  blackboard)  was  easy  to 
operate. 

25.  I  would  take  another  course  which  used  this  delivery  system. 

26.  There  should  be  more  interaction  among  the  sites. 

NOTE:  WPAFB  personnel  do  not  answer  questions  27*30,  go  to  Part  II. 

27.  The  absence  of  eye  contact  with  the  instructor  created  a  learning 
barrier/problem. 

28.  The  person  locally  aiding  the  course  director  (site  monitor)  appeared 
knowledgeable  of  equipment  operation. 

29.  The  local  person  aiding  the  course  director  (site  monitor)  had  the 

room  and  materials  prepared  for  class. 

« 

_  30.  A  subject  matter  expert  should  be  available  at  each  site  to  assist 
students. 

PART  1 1 

Please  answer  the  questions  on  the  back  of  your  answer  sheet. 

(NOTE:  Students  were  asked  to  respond  to  open-ended  ques¬ 
tions  in  Part  II.  Student  responses  constitute  the  written 
comments  referred  to  throughout  the  research.  Due  to  the 
nature  of  the  responses,  they  were  not  included  in  the  for¬ 
mal  data  analysis. 

The  nonTEDS  version  included  questions  1-20.) 
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APPENDIX  C 

INSTRUCTOR  CRITIQUE  OF  TELETEACH 
DELIVERY  SYSTEM 
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Use  the  attached  answer  sheet  to  mark  your  responses.  Please  do  NOT  write 
your  name  or  social  security  number  anywhere  on  the  answer  sheet.  Select 
only  one  answer  to  each  question.  Use  only  a  No.  2  pencil  when  filling 
out  the  answer  sheet.  DO  NOT  USE  INK.  Mark  the  answer  sheet  carefully  to 
negate  computer  error.  Fill  in  the  box  with  a  heavy  mark;  do  not  go  out¬ 
side  the  lines  of  the  box.  If  you  make  a  mistake,  erase  the  mark  completely 
before  entering  a  new  one.  The  last  statement  on  the  critique  requires  a 
written  response.  Put  your  answer  on  the  back  of  the  answer  sheet.  Addi¬ 
tional  comments  are  welcome. 


Respond  by  using  the  options  A  thru  E  indicating  the  degree  to  which  you 
agree  with  the  statements  below  (1-10). 

A.  Strongly  agree 

B.  Agree 

C.  Neither  agree  nor  disagree 
0.  Disagree 

E.  Strongly  disagree 

1.  Before  using  the  teleteach  system,  I  would  have  I  i ked  more  explanation  of 
its  particular  demands  upon  me  as  an  instructor. 

2.  There  were  equipment  and/or  transmission  problems  during  my  presentation. 

3.  I  feel  students  at  WPAFB  tend  to  be  slighted  when  the  Teleteach  system  is 
used. 


A.  Student  participation  from  remote  sites  met  or  exceeded  my  expectations. 

5.  I  tried  special  techniques  to  increase  remote  student  involvement. 

6.  My  inability  to  discuss  the  subject  face  to  face  with  remote  students  was 
very  disturbing. 

7.  My  inability  to  see  students  at  the  remote  sites  was  very  disturbing. 

8.  I  would  like  to  teach  again  using  the  Teleteach  system. 

9.  After  using  the  Teleteach  system  I  feel  more  favorable  towards  its  use. 

10.  Advantages  of  the  system  outweigh  the  disadvantages. 

11.  The  Teleteach  system  caused  me  to  change  my  presentation. 

A.  A  great  deal 

B.  Some 

C.  Not  at  al I 


NOTE:  Please  explain  on  reverse  side  of  answer  sheet. 


12.  If  I  were  to  teach  again  using  the  Teleteach  system,  I  would  change 
my  presentation, 

A.  A  great  deal 

B.  Some 

C.  Not  at  at  1 

NOTE:  Please  explain  on  reverse  side  of  answer  sheet. 

13.  I  have  given  my  presentation  in  this  course  WITHOUT  teleteach. 

A.  Never  before 

B.  Once  before 

C.  Twice  before 

0.  Three  times  before 
E.  More  than  3  times  before 

1 k.  I  have  given  my  presentation  in  this  course  USING  the  Teleteach  system. 

A.  Never  before 

B .  Once  before 

C.  Twice  before 

0.  Three  times  before 
E.  More  than  3  times  before 

15.  WPAFB  students  discussed  aspects  of  my  teleteach  presentation  with  me 
(before  and  after  class). 

A.  Frequently 
3.  Seldom 
C.  Never 

16.  The  Electronic  Blackboard  is  an  essential  component  of  the  delivery  system. 

A.  Strongly  Agree 

B.  Agree 

C.  Not  sure 

D.  Disagree 

E.  Strongly  disagree 

17'  I  used  the  Electronic  Blackboard.,. 

A.  Frequently 

B.  Some 

C.  Never 


Please  complete  the  reverse  side  of  the  answer  sheet. 
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